Richardson extrapolation

Suppose every estimate you compute is wrong by an amount that shrinks in a known, predictable way as you make your step smaller — say the error roughly quarters each time you halve the step. Then two estimates at two step sizes contain enough information to FIGURE OUT and subtract most of that error, producing a far better number than either input. Richardson extrapolation is this trick of combining cheap, low-order estimates to manufacture a high-order one.

Here is the engine. Suppose a method A(h) approximates a true value L with an error expansion A(h) = L + c h^p + (higher-order terms). Compute it at two steps, A(h) and A(h/2). The errors are c h^p and c (h/2)^p = c h^p / 2^p. Form the weighted combination (2^p A(h/2) - A(h)) / (2^p - 1): the c h^p terms are engineered to cancel exactly, leaving an error of the NEXT, higher order. For the central-difference derivative p = 2, so the combination (4 A(h/2) - A(h)) / 3 jumps from O(h^2) to O(h^4) accuracy. Repeat the cancellation on the new, higher-order estimates and you climb an entire ladder of accuracy — this is exactly how Romberg integration turns the humble trapezoidal rule into a high-order quadrature.

Richardson extrapolation is one of the most powerful and reusable ideas in numerical analysis: it costs almost nothing (a couple of arithmetic operations) yet can lift a sluggish method several orders. The honest fine print is that it RELIES on the assumed error expansion being correct: the method must really have error c h^p + ... in even or known powers, and the function must be smooth enough for those higher derivatives to exist. If the integrand has a kink or singularity, or if round-off has already corrupted the estimates, the assumed expansion breaks and extrapolation can actually make things worse, not better.