Moore-Penrose pseudoinverse
Most matrices have no inverse — they are non-square, or square but singular. Yet you still want to solve Ax = b as well as possible. The Moore-Penrose pseudoinverse A^+ is the matrix that does exactly that: it is the best-fit inverse, the one that returns the least-squares solution when the system is overdetermined and the minimum-norm solution when it is underdetermined.
The SVD builds it almost for free. If A = U Sigma V^T, then A^+ = V Sigma^+ U^T, where Sigma^+ is formed by transposing Sigma and replacing each nonzero singular value sigma_i by 1/sigma_i (leaving the zeros as zeros). You invert what is invertible and ignore what is not. When A is actually invertible, A^+ collapses to the ordinary A^-1, so it is a true generalization.
What does x = A^+ b actually give you? Among all x that minimize ||Ax - b|| (the least-squares fit to an inconsistent system), it picks the one with the smallest ||x||. That double optimality — best fit, then smallest norm to break ties — is what makes A^+ the canonical solver. It is uniquely pinned down by the four Penrose conditions: A A^+ A = A, A^+ A A^+ = A^+, and (A A^+) and (A^+ A) are both symmetric.
A practical warning: tiny singular values become huge 1/sigma_i values, so A^+ can wildly amplify noise. That is why in practice one often uses a truncated pseudoinverse, zeroing out singular values below a threshold — trading exactness for numerical stability.
Invert the nonzero singular values, leave the zeros alone, and swap U and V.
For full-column-rank A, the pseudoinverse equals the normal-equations formula (A^T A)^-1 A^T, but computing it through the SVD is far more numerically stable than forming A^T A.