Inner products bring back geometry
An inner product on a complex vector space H is a map ⟨·,·⟩ that is linear in the first slot, conjugate-symmetric (⟨y, x⟩ = conjugate of ⟨x, y⟩), and positive-definite (⟨x, x⟩ > 0 for x ≠ 0). It induces a norm ||x|| = sqrt(⟨x, x⟩). A Hilbert space is an inner-product space that is complete in this norm — a Banach space whose norm happens to come from an inner product. That extra structure is everything: it gives us orthogonality.
Cauchy–Schwarz: |⟨x, y⟩| ≤ ||x|| ||y||.
Proof (real case; assume y ≠ 0). For every real t,
0 ≤ ||x - t y||^2 = ⟨x - t y, x - t y⟩
= ||x||^2 - 2 t ⟨x, y⟩ + t^2 ||y||^2.
This quadratic in t is ≥ 0 for all t, so it has at most one
real root; its discriminant is ≤ 0:
(2 ⟨x, y⟩)^2 - 4 ||y||^2 ||x||^2 ≤ 0
⇒ ⟨x, y⟩^2 ≤ ||x||^2 ||y||^2
⇒ |⟨x, y⟩| ≤ ||x|| ||y||. ∎
Consequence — the triangle inequality for ||·||:
||x + y||^2 = ||x||^2 + 2⟨x, y⟩ + ||y||^2
≤ ||x||^2 + 2||x|| ||y|| + ||y||^2
= (||x|| + ||y||)^2.
So ||x + y|| ≤ ||x|| + ||y||: the inner product really does
give a norm.Orthonormal bases and projection
Two vectors are orthogonal when ⟨x, y⟩ = 0; then Pythagoras holds: ||x + y||^2 = ||x||^2 + ||y||^2. A family (e_n) is an orthonormal basis if ⟨e_m, e_n⟩ = 0 for m ≠ n, ||e_n|| = 1, and the e_n span a dense subspace. For such a basis every x has a Fourier expansion x = sum_n ⟨x, e_n⟩ e_n, the coefficients satisfy Bessel's inequality sum |⟨x, e_n⟩|^2 ≤ ||x||^2, and when the basis is complete this is an equality — Parseval's identity.
The geometric heart is the projection theorem: if M is a closed subspace of a Hilbert space H, then every x splits uniquely as x = p + q with p in M and q in the orthogonal complement M⊥. The vector p is the unique closest point of M to x, and it is characterized by x − p being orthogonal to all of M. This is the abstract reason least-squares approximation works.
Best-approximation characterization of the projection.
Let M be a subspace, p ∈ M. Claim:
||x - p|| ≤ ||x - m|| for all m ∈ M
⇔ ⟨x - p, m⟩ = 0 for all m ∈ M.
(⇐) Suppose x - p ⊥ M. For any m ∈ M, p - m ∈ M, so
||x - m||^2 = ||(x - p) + (p - m)||^2
= ||x - p||^2 + ||p - m||^2 (Pythagoras)
≥ ||x - p||^2.
So p minimizes the distance. Equality forces p = m, giving
uniqueness.
(⇒) Suppose p minimizes. Fix m ∈ M, real t, write
g(t) = ||x - p - t m||^2
= ||x - p||^2 - 2 t ⟨x - p, m⟩ + t^2 ||m||^2.
g has its minimum at t = 0, so g'(0) = -2⟨x - p, m⟩ = 0,
i.e. ⟨x - p, m⟩ = 0. (Repeat with i·m for the complex case.) ∎