geometry of the SVD

Here is the single most illuminating fact about linear maps: no matter how tangled a matrix looks, every one of them does the same three things in sequence. Rotate, stretch along the coordinate axes, rotate again. The SVD A = U Sigma V^T is literally this sentence written as matrices — V^T rotates, Sigma stretches, U rotates.

Follow a vector x through. First V^T spins x so that the right singular vectors line up with the coordinate axes (an orthogonal map, pure rotation/reflection, no distortion). Then Sigma stretches axis i by the factor sigma_i (and, if A is non-square, drops or pads dimensions). Finally U rotates the stretched result into the output space. Rotation, stretch, rotation — that is all any linear map ever does.

The cleanest picture is what A does to the unit sphere. Spheres are unchanged by the two rotations, so the entire shaping happens in the stretch step: the sphere becomes an ellipsoid. The directions of that ellipsoid's axes are the left singular vectors u_i, and the lengths of its semi-axes are the singular values sigma_i. The whole behavior of A is captured by that one ellipsoid — orientation and stretch in a single image.

This geometry quietly explains everything else in the field. The longest semi-axis sigma_1 is the spectral norm (max stretch); a zero semi-axis is a collapsed direction, i.e. the null space; the ratio of longest to shortest is the condition number; flattening the smallest axes is low-rank approximation. Once you see the rotate-stretch-rotate ellipsoid, the rest of SVD theory is just reading details off that picture.