projection theorem
The projection theorem says you can always drop a perpendicular. Given a point and a closed subspace inside a Hilbert space, there is exactly one point of the subspace nearest to your point, and the line from your point to it meets the subspace at a right angle — just like dropping a perpendicular from a point to a plane in ordinary geometry. This single geometric fact powers least-squares approximation and Fourier expansion alike.
Precisely, let M be a closed subspace of a Hilbert space H and let x be any vector. Then there is a unique m in M minimizing the distance ||x - m||, and it is characterized by the orthogonality condition x - m belongs to M-perp. Consequently H = M (+) M-perp, and the map P sending x to m — orthogonal projection onto M — is a bounded linear operator with ||P|| = 1 (for M nonzero), P^2 = P, and the Pythagorean split ||x||^2 = ||P x||^2 + ||x - P x||^2.
Two hypotheses are essential and not decoration. Completeness of H supplies the limit needed to realize the minimizing vector; closedness of M ensures that limit lands inside M. Drop closedness and the nearest point can fail to exist; the infimum of distances is approached but never attained, which is exactly why “closed subspace,” not merely “subspace,” is the standing assumption.
Approximating f(x) = x on [-pi, pi] by the closed subspace M spanned by 1, cos x, sin x, the best approximant is the orthogonal projection, computed by Fourier coefficients: the projection is 2 sin x, since <x, sin x>/<sin x, sin x> = 2 while the constant and cosine coefficients vanish by symmetry. The error x - 2 sin x is orthogonal to all of M.
Best least-squares fit as an orthogonal projection; the residual is perpendicular to the fitting space.
Existence of the minimizer rests on the parallelogram law: a minimizing sequence in a closed convex set is Cauchy, hence convergent in a complete space. So the proof secretly uses the inner-product structure twice — once for orthogonality, once for the parallelogram identity that gives the Cauchy estimate.