the reproducing kernel Hilbert space
Every covariance function K secretly carries a Hilbert space of functions inside it — the reproducing kernel Hilbert space (RKHS) of K. This space is the right notion of 'directions you are allowed to shift a Gaussian process in' and is the algebraic skeleton behind the Cameron-Martin theorem, large deviations for Gaussians, and kernel methods in machine learning. It converts the analytic object K(s, t) into a geometry of functions.
Concretely, start from the kernel functions K_t := K(., t), one for each index t. Define an inner product by declaring < K_s, K_t > = K(s, t), extend by linearity to finite combinations sum a_i K_(t_i), and complete to get a Hilbert space H_K. By construction the kernel reproduces evaluation: for any f in H_K, < f, K_t > = f(t). That single 'reproducing property' is what names the space. Equivalently (Moore-Aronszajn theorem) every symmetric positive-semidefinite K determines a UNIQUE RKHS of which it is the reproducing kernel, and vice versa. The norm ||f||_(H_K) measures how 'aligned' f is with the kernel's geometry; functions of large norm are penalized, which is exactly the smoothness/regularization story in statistics.
Why it matters: the RKHS of a covariance is, up to isometry, the Cameron-Martin space of the associated Gaussian measure — the set of admissible shifts under which the law is only re-weighted, never destroyed. Crucially, the RKHS is much SMALLER than the space the process lives in: for Brownian motion on [0, 1] the RKHS consists of absolutely continuous functions h with h(0) = 0 and finite integral of h'(t)^2 dt (norm = the L^2 norm of h'), whereas Brownian paths themselves are nowhere differentiable and have probability ZERO of lying in their own RKHS. This 'the process never visits its Cameron-Martin space' paradox is fundamental, not a defect.
For Brownian motion, K(s, t) = min(s, t). Its RKHS is the Cameron-Martin space H = { h : h(0) = 0, h absolutely continuous, integral of h'(t)^2 dt < infinity }, with < g, h >_H = integral g'(t) h'(t) dt. One checks the reproducing property: < h, K_t >_H = integral_0^1 h'(u) (1_(u <= t)) du = h(t), since K_t(u) = min(u, t) has derivative 1 on [0, t].
The RKHS of min(s, t) is the space of finite-energy paths starting at 0 — exactly Brownian motion's Cameron-Martin space.
The RKHS is a measure-zero subset of the path space: the Gaussian process almost surely does NOT lie in its own RKHS, even though that RKHS controls all of its geometry.