Size as maximum stretch
Given a vector norm, the induced (operator) norm of a matrix A is its worst-case stretch factor: ||A|| = max over v != 0 of ||A v|| / ||v||. Equivalently, the largest ||A v|| over all unit vectors v. This is the most a matrix can amplify any input.
The three induced norms you can compute by hand
For the three p-norms, the induced matrix norm has a closed form. ||A||_1 is the largest absolute column sum, ||A||_inf is the largest absolute row sum, and ||A||_2 is the largest singular value sigma_max from the singular value decomposition (the hardest of the three to compute by hand).
A = [ 1, -7;
4, 2 ]
||A||_1 = max( |1|+|4| , |-7|+|2| ) = max(5, 9) = 9 (column sums)
||A||_inf = max( |1|+|-7| , |4|+|2| ) = max(8, 6) = 8 (row sums)
||A||_2 = sigma_max(A) ~= 7.34 (largest singular value)
Sanity: A maps the unit-2-ball to an ellipse whose longest
semi-axis has length sigma_max ~= 7.34.Submultiplicativity: errors that chain
Every induced norm obeys submultiplicativity: ||A B|| <= ||A|| * ||B||. The proof is one line of the compatibility bound: ||A B v|| <= ||A|| * ||B v|| <= ||A|| * ||B|| * ||v||, then take the max over unit v. This is what lets us bound the effect of a product or a power of perturbed matrices.