QR decomposition (revisited)

You met A = Q R in a first course as the matrix form of Gram-Schmidt: Q has orthonormal columns, R is upper triangular. The columns of Q are an orthonormal basis for the column space of A built up column by column. That picture is correct, but the computation behind it deserves a second look.

Classical Gram-Schmidt is numerically fragile. Subtracting off projections accumulates rounding error, and the computed columns of Q drift away from orthogonality, sometimes badly. The modern way to compute QR does not orthogonalize the columns of A at all. Instead it applies a sequence of orthogonal transformations to A from the left, zeroing out the subdiagonal one column at a time, until what remains is R; the accumulated transformations form Q.

Those orthogonal transformations are Householder reflections (the default, reflecting whole columns) or Givens rotations (one entry at a time, ideal for sparse or already-nearly-triangular matrices). Because every step is an exact orthogonal map, lengths and angles are preserved and rounding errors do not amplify. The result is backward stable: the computed Q and R factor a matrix very close to A.

QR is the right tool for least squares. Solving min ||A x - b|| via the normal equations squares the condition number, but factoring A = Q R and solving R x = Q^T b avoids that squaring entirely. QR also drives the QR algorithm for eigenvalues, where it is iterated to push a matrix toward triangular form.