MINRES

MINRES is the right tool for a matrix that is symmetric but not positive-definite — symmetric indefinite, meaning its eigenvalues are real (thanks to symmetry) but some are positive and some negative. Such matrices appear all over: saddle-point systems from constrained optimization and fluid mechanics, certain shifted systems A - sigma I, and Helmholtz-type wave problems. Conjugate gradient cannot be trusted here because its energy bowl is no longer a bowl (an indefinite matrix gives a saddle, not a minimum), so CG can break down. MINRES fills exactly this gap.

Like GMRES, MINRES minimizes the residual norm ||b - A x|| over the Krylov subspace — that is what the name means, MINimal RESidual. The crucial difference is that it exploits symmetry. For a symmetric matrix the Arnoldi process collapses into the much cheaper Lanczos process, in which each new basis vector needs to be orthogonalized only against the previous TWO (a three-term recurrence), not against the entire history. The consequence is wonderful: MINRES gets GMRES's residual-minimizing optimality but at constant work and constant memory per step — it stores only a fixed handful of vectors no matter how many iterations it runs. It is, in effect, the symmetric-matrix version of GMRES with the runaway cost removed.

So the decision tree is clean. If A is symmetric and positive-definite, use conjugate gradient (cheaper still, and minimizes the natural energy norm). If A is symmetric but indefinite, use MINRES — it is robust where CG fails, and unlike GMRES it never needs restarting. If A is nonsymmetric, neither applies and you reach for GMRES or BiCGStab. One honest caveat shared with all these short-recurrence Krylov methods: in floating-point arithmetic the Lanczos vectors gradually lose orthogonality, which can slow convergence relative to the clean theory, though MINRES is generally well-behaved in practice. And, as ever, conditioning sets the speed and preconditioning is what accelerates it.