Orthonormal systems: a basis you can sum but never finish

From finite orthonormal bases to infinite systems

Recall the cleanest fact of Vol I geometry: if {q_1, ..., q_n} is an orthonormal basis, then every vector is v = sum <v, q_k> q_k, with the coordinates read off by inner products. The coefficients <v, q_k> are the projections onto each axis. We want the same formula when there are infinitely many q_k.

An orthonormal system {e_k} in a Hilbert space is a (possibly infinite) set with <e_i, e_j> = 0 for i != j and ||e_k|| = 1. The numbers c_k = <x, e_k> are called the Fourier coefficients of x with respect to the system — the direct generalization of coordinates. The question is whether the infinite sum sum c_k e_k converges, and whether it reconstructs x.

Bessel, Parseval, and what "complete" means

Two inequalities organize everything. Bessel's inequality says sum |c_k|^2 <= ||x||^2 for *any* orthonormal system: the projected energy never exceeds the total energy. The system is a complete orthonormal system precisely when equality holds for every x — that is Parseval's identity, sum |c_k|^2 = ||x||^2 — meaning no energy is lost and nothing is missing from the system.

The classic complete orthonormal system in L^2[-pi, pi]:

  e_n(t) = (1/sqrt(2*pi)) * exp(i n t),   n = ..., -1, 0, 1, ...

  orthonormal:  <e_m, e_n> = (1/2pi) integral exp(i(m-n)t) dt
                           = 1 if m = n,  0 otherwise.

Fourier coefficients of f:   c_n = <f, e_n>
Fourier series (reconstruction):   f = sum_n  c_n e_n

Parseval (no energy lost):   integral |f|^2  =  sum_n |c_n|^2

Why it is COMPLETE, not just orthonormal:
  the ONLY function orthogonal to every e_n is f = 0.
  (No nonzero direction is missed -> the system spans, in the
   closure sense, all of L^2.)

Projection: best approximation that always exists

The single most useful theorem of Hilbert geometry is the projection theorem: if M is a *closed* subspace, every x has a unique closest point Px in M, and the error x - Px is orthogonal to all of M. This is the least-squares idea of Vol I, now guaranteed in infinite dimensions — provided M is closed, which is where completeness earns its keep.

1. Pick an orthonormal system {e_1, ..., e_N} spanning the closed subspace M (or use Gram-Schmidt to build one).
2. Compute Fourier coefficients c_k = <x, e_k> for k = 1..N.
3. The projection is Px = sum_{k=1}^N c_k e_k — the best approximation of x inside M in the norm of the space.
4. The residual x - Px is orthogonal to every e_k, and ||x||^2 = ||Px||^2 + ||x - Px||^2 (Pythagoras still holds).

Truncating a Fourier series to its first N terms is *exactly* this projection onto the span of e_1, ..., e_N — which is why a partial Fourier sum is the best N-term trigonometric approximation in the L^2 sense. The infinite-dimensional picture has handed us back the most practical tool of finite-dimensional geometry, completely intact.