Matrix Norms & Perturbation

matrix norm

A matrix norm measures the size of a matrix with one number, just as a vector norm sizes a vector. Since the m-by-n matrices themselves form a vector space, every vector-norm axiom still applies. But matrices also multiply, and that extra structure makes us want one more property.

That property is submultiplicativity: ||A B|| <= ||A|| ||B|| whenever the product is defined. A norm that satisfies all the vector axioms plus this product rule is called a (consistent) matrix norm. It is exactly what lets you control errors that chain through repeated products, the bread and butter of numerical analysis.

Two big families dominate. The Frobenius norm ||A||_F = sqrt(sum_{i,j} |a_{ij}|^2) simply treats the matrix as one long vector and takes its Euclidean length; it equals sqrt(sum sigma_i^2), the root-sum-square of the singular values. The operator (induced) norms instead measure how much A can stretch a vector. Both are submultiplicative, but they answer different questions: Frobenius is the total energy, the operator 2-norm is the worst-case stretch sigma_max.

A caveat worth remembering: not every natural-looking norm is submultiplicative. The entrywise max norm max |a_{ij}| obeys the vector axioms but can fail ||AB|| <= ||A|| ||B||, so it is a norm on the space but not a matrix norm in the strict sense. Always check which guarantee you actually have.

A = [3, 0; 0, 4]: ||A||_F = 5, ||A||_2 = 4

Frobenius sums all singular energy (sqrt(9+16)=5); the operator 2-norm reports only the largest stretch (4).

Frobenius and the operator 2-norm both equal sigma_max only when A has rank 1. In general ||A||_2 <= ||A||_F <= sqrt(rank A) ||A||_2, so Frobenius is the easier-to-compute upper proxy for the operator norm.

Also called
Frobenius normsubmultiplicative norm