singular value decomposition

The singular value decomposition writes ANY matrix A (square or not, even a tall or wide one) as A = U*S*V^T. Here U and V are orthogonal matrices (rotations or reflections), and S is diagonal holding the nonnegative singular values, sorted from largest to smallest. It is the most general and arguably the most important decomposition in all of linear algebra.

Geometrically it tells a beautiful story: every linear map, no matter how tangled it looks, is just three simple steps in disguise. First rotate (V^T), then stretch each axis by its singular value (S), then rotate again (U). The singular values measure how much the map stretches space along each of its principal directions.

Because it works for every matrix and exposes this clean structure, SVD is the foundation of many applications: it underlies low-rank approximation and image compression, principal component analysis, the condition number, and robust ways to solve least-squares problems. If you only learn one decomposition deeply, this is the one.