Spectral radius vs. norm
The spectral radius rho(A) is the largest |eigenvalue| of A. It always sits below any induced norm: rho(A) <= ||A||. But it is the norm that controls a single application, while rho(A) controls the long run — Gelfand's theorem says the spectral radius (Gelfand) formula rho(A) = lim ||A^k||^(1/k).
The consequence is sharp: A^k -> 0 as k -> infinity if and only if rho(A) < 1. A matrix can have rho(A) < 1 yet ||A|| > 1, so its powers may grow transiently before they decay — the source of much numerical surprise.
The Neumann series
When rho(E) < 1, the Neumann series gives a clean inverse: (I - E)^-1 = I + E + E^2 + E^3 + ... — the matrix analogue of the geometric series 1/(1-x). Submultiplicativity also yields the bound ||(I - E)^-1|| <= 1/(1 - ||E||) whenever ||E|| < 1.
This is exactly what lets us perturb an invertible matrix: if A is invertible and the perturbation is small, A + dA stays invertible and we can bound the change in its inverse. The Neumann bound is the bridge from norms to the condition number.
Condition number: the amplification factor
For solving A x = b, the condition number kappa(A) = ||A|| * ||A^-1|| (extending Vol I's condition number) is the worst-case factor by which relative input error is amplified into relative output error: ||dx||/||x|| <= kappa(A) * ||db||/||b||. In the 2-norm, kappa_2(A) = sigma_max / sigma_min.
A = [ 1.000, 1.000;
1.000, 1.001 ]
sigma_max ~= 2.0005, sigma_min ~= 0.0005
kappa_2(A) = sigma_max / sigma_min ~= 4002
b = (2.000, 2.001)^T -> x = (1, 1)^T
b+db = (2.000, 2.002)^T -> x' = (0, 2)^T
relative input change ||db||/||b|| ~= 0.00035
relative output change ||dx||/||x|| ~= 1.0
amplification observed ~= 2850 (kappa bounds it: <= 4002)