preconditioning

Iterative solvers converge fast on well-conditioned, eigenvalue-clustered matrices and crawl on ill-conditioned ones. Preconditioning is the move that bends the second kind into the first: instead of solving Ax = b, you solve an equivalent system M^-1 A x = M^-1 b, where M is chosen so that M^-1 A is much nicer — its eigenvalues bunched together near 1, its condition number small. The solution is unchanged; only the geometry the iteration walks through improves.

The art is the trade-off built into M. The ideal preconditioner would be M = A itself, making M^-1 A = I and converging in one step — but applying M^-1 then costs as much as solving the original problem. So a good M must be two things at once: a decent approximation of A (so M^-1 A is well-conditioned) and cheap to apply (so each iteration stays affordable). Every preconditioner is a compromise between those poles.

Common choices span that spectrum. Jacobi (diagonal) and Gauss-Seidel preconditioners are nearly free but weak. Incomplete factorizations like ILU or incomplete Cholesky drop the fill-in that a full factorization would create, giving a stronger M at moderate cost. Multigrid and domain-decomposition preconditioners are sophisticated and, for the PDEs they target, can make iteration counts nearly independent of problem size.

It is hard to overstate how decisive this is. For the large sparse systems that dominate scientific computing, the choice of preconditioner usually matters far more than the choice of Krylov method. A mediocre solver with a great preconditioner routinely beats a great solver with none. Picking M well is the single biggest lever on iterative performance.