Neumann series
The Neumann series is the matrix version of the geometric series 1/(1 - r) = 1 + r + r^2 + ... . It expresses the inverse of I - A as an infinite sum of powers: (I - A)^-1 = I + A + A^2 + A^3 + ... . When the series converges, it both proves the inverse exists and gives a constructive way to approximate it.
Convergence has a clean sufficient condition: if ||A|| < 1 in any submultiplicative norm, the partial sums form a Cauchy sequence (since ||A^k|| <= ||A||^k decays geometrically), so the series converges and its sum is (I - A)^-1. The sharp condition is spectral: the series converges if and only if rho(A) < 1, the spectral radius being below one.
The practical use is perturbation of the inverse. If you know B^-1 and perturb B slightly to B + E, write B + E = B(I + B^-1 E) and expand: (B + E)^-1 = (I + B^-1 E)^-1 B^-1 = (I - B^-1 E + ...) B^-1, valid when ||B^-1 E|| < 1. This is how you estimate how the solution of a linear system shifts under a small change in the matrix.
It also bounds the conditioning of inversion directly. From the series you get ||(I - A)^-1|| <= 1 / (1 - ||A||), which quantifies how close to singular I - A can drift before its inverse blows up — the analytic engine behind condition-number estimates and the convergence of iterative solvers.
Convergent when ||A|| < 1; the bound shows the inverse blows up as ||A|| approaches 1 (I - A nears singular).
The norm condition ||A|| < 1 is sufficient but not necessary; the spectral condition rho(A) < 1 is exact. A non-normal A can have rho(A) < 1 yet ||A|| > 1, so the series still converges even though the simple norm test fails.