Gram-Schmidt orthogonalization
/ GRAM-shmit /
Suppose you have a list of building blocks that span what you want but are not perpendicular — they lean on each other. Gram-Schmidt is the systematic way to straighten them: it takes any independent set and produces an orthogonal set spanning exactly the same space, by repeatedly subtracting off the parts that overlap with what you have already cleaned up. The same procedure that orthogonalizes vectors orthogonalizes functions, once you fix a (possibly weighted) inner product.
The mechanism is a running subtraction. Keep the first element. For each next element, subtract its projection onto every orthogonal element built so far; what remains is, by construction, orthogonal to all of them. In a weighted function space with inner product (f, g) = integral of f g w dx, you orthogonalize a sequence like 1, x, x^2, x^3, ... by setting each new polynomial equal to the next power minus its projections onto the previous orthogonal ones. Feed in the weight w = 1 on [-1, 1] and you generate the Legendre polynomials; feed in w = e^{-x^2} on the whole line and you get the Hermite polynomials; w = e^{-x} on [0, infinity) gives Laguerre. The classical orthogonal polynomials are literally Gram-Schmidt applied to the monomials with the right weight.
This is why orthogonal-polynomial bases are not magic handed down from nowhere — they are constructed, and you can construct your own on a custom interval with a custom weight whenever a problem demands it. In numerical work, though, the textbook (classical) Gram-Schmidt is notoriously sensitive to round-off, so practitioners use the modified Gram-Schmidt variant or a QR factorization; the mathematics is the same but the arithmetic is more stable.
Orthogonalize 1, x, x^2 on [-1, 1] with weight 1. Start with q_0 = 1. Then q_1 = x minus its projection on q_0, which is 0, so q_1 = x. Then q_2 = x^2 minus (projection on q_0) minus (projection on q_1) = x^2 - 1/3. These are proportional to P_0, P_1, P_2 — the first Legendre polynomials.
Gram-Schmidt on the monomials with weight 1 on [-1, 1] reproduces the Legendre polynomials up to normalization.
Gram-Schmidt gives an orthogonal set whose span equals the original span, but it is not unique: change the order of the inputs, the interval, or the weight, and you get a different orthogonal family.