existence of the SVD

The headline theorem of this whole field is breathtakingly general: every matrix, with no exceptions, can be factored as A = U Sigma V^T. Not just square ones, not just invertible ones, not just diagonalizable ones — any m-by-n real matrix at all. This is what makes the SVD the single most universal decomposition in linear algebra.

Here is what the factors are. U is an m-by-m orthogonal matrix whose columns are the left singular vectors; V is an n-by-n orthogonal matrix whose columns are the right singular vectors; and Sigma is an m-by-n diagonal-shaped matrix with the singular values sigma_1 >= sigma_2 >= ... >= 0 on its main diagonal and zeros everywhere else. (Over the complex numbers, U and V are unitary and V^T becomes V^*.)

Why does it always exist? The matrix A^T A is symmetric and positive semidefinite, so the spectral theorem hands you an orthonormal eigenbasis with nonnegative eigenvalues — those become V and the squared singular values. Setting u_i = (1/sigma_i) A v_i for the nonzero sigma's gives orthonormal left vectors, and you extend to a full orthonormal basis for U. The construction never fails because the spectral theorem never fails for symmetric matrices.

The payoff is that anything you want to know about A — its rank, its norm, its four subspaces, its best low-rank approximations, its pseudoinverse — is read straight off these three factors. Where eigenvalue methods stall on non-square or defective matrices, the SVD always delivers an orthonormal, diagonal core.