projection theorem

The projection theorem is the reason geometry still works in infinite dimensions. In R^n you can drop a perpendicular from any point onto any subspace and split the point into a part inside plus a part perpendicular. The projection theorem says this best-approximation-by-a-foot-of-perpendicular survives in any Hilbert space — provided the subspace is closed. That one word, closed, is the new admission ticket.

Precisely: let M be a closed subspace of a Hilbert space H. Then every x in H has a unique decomposition x = m + n with m in M and n in the orthogonal complement M-perp, and m is the unique nearest point of M to x. Symbolically H = M (+) M-perp as an orthogonal direct sum, and (M-perp)-perp = M. The map x -> m is the orthogonal projection onto M, a self-adjoint idempotent of norm 1 (unless M is trivial).

Why closedness is non-negotiable: in infinite dimensions a subspace can be dense without being everything, and the nearest point of a non-closed subspace may simply fail to exist — the perpendicular foot lands on a hole. Closedness guarantees the infimum distance is actually attained. Completeness of H is what makes the minimizing sequence converge, so this theorem fails in incomplete inner-product spaces.

Why it matters: this is the engine behind least squares, conditional expectation in probability (which is literally orthogonal projection in L^2), Fourier partial sums (projection onto the span of finitely many basis functions, the best L^2 approximation), and the existence of orthogonal complements that makes the whole Hilbert-space toolkit work. It is, quite simply, the theorem that lets you say best fit and mean it.