the Gaussian process and its covariance function
A Gaussian process is the infinite-dimensional version of the Gaussian (normal) random vector you met in Volume I. Where a Gaussian vector in R^n is described entirely by its mean vector and covariance matrix, a Gaussian process X = (X_t)_(t in T) is a collection of random variables, one for each index t in some set T, such that every finite subcollection (X_(t_1), ..., X_(t_n)) is jointly Gaussian. The miracle is that this huge, possibly continuum-indexed object is still pinned down completely by just two functions: a mean and a covariance.
Precisely, X is Gaussian if for every finite set of indices t_1, ..., t_n the vector (X_(t_1), ..., X_(t_n)) is a multivariate normal. Its law is then determined by the mean function m(t) = E[X_t] and the covariance function K(s, t) = Cov(X_s, X_t) = E[(X_s - m(s))(X_t - m(t))]. Any covariance function K must be symmetric, K(s, t) = K(t, s), and positive semidefinite: for all choices of points and real coefficients a_i, the double sum of a_i a_j K(t_i, t_j) is >= 0. This is exactly the condition that every finite covariance matrix be a genuine (non-negative) covariance matrix. Conversely, by Kolmogorov's existence theorem, any symmetric positive-semidefinite K and any mean m arise from some Gaussian process. One almost always takes m = 0 (a centered Gaussian process), since the mean is a deterministic shift carrying no probabilistic structure.
Why it matters: the Gaussian world is the most tractable corner of probability in infinite dimensions, because the covariance function carries all the information. Brownian motion is the Gaussian process with K(s, t) = min(s, t); the Brownian bridge has K(s, t) = min(s, t) - st; the Ornstein-Uhlenbeck process is stationary Gaussian with K(s, t) = exp(-|t - s|). Smoothness of paths, the size of the supremum, and concentration all turn out to be governed by K alone. One honest caution: jointly Gaussian is much stronger than each X_t being marginally Gaussian — uncorrelated jointly Gaussian variables are independent, but uncorrelated merely-marginally-Gaussian variables need not be.
Standard Brownian motion B_t on [0, 1] is the centered Gaussian process with covariance K(s, t) = min(s, t). Check: Var(B_t) = min(t, t) = t, and for s < t, Cov(B_s, B_t) = E[B_s B_t] = E[B_s(B_t - B_s)] + E[B_s^2] = 0 + s = min(s, t). The whole law of Brownian motion is encoded in that one little function.
A Gaussian process is determined by its mean and covariance; for Brownian motion the covariance is simply min(s, t).
Marginal Gaussianity of each X_t is far weaker than joint Gaussianity of the whole family; only the latter defines a Gaussian process and makes uncorrelated equal independent.