low-rank approximation

Low-rank approximation replaces a big, complicated matrix with a much simpler one that is almost as good. The recipe comes straight from the SVD A = U*S*V^T: the singular values in S are sorted from largest to smallest, so you keep only the largest few and discard the tiny ones. The result is a matrix of low rank that captures the dominant structure while needing far less to store.

A famous theorem (Eckart-Young) makes this precise: truncating to the top k singular values gives the best possible rank-k approximation, the one closest to the original. The discarded small singular values were contributing little, so dropping them costs only a little accuracy.

This is the engine behind image compression (a photo's matrix of pixels often has only a few significant singular values) and recommender systems (a sparse table of user ratings is approximated by a thin product that fills in the gaps). The honest trade-off: you sacrifice some fine detail in exchange for big savings in storage and computation, and you must choose k to balance the two.