backward error analysis
/ Wilkinson: WIL-kin-sun /
Before the 1950s, analysing rounding errors meant chasing every tiny error forward through a calculation and watching the bound balloon into something useless — predicting catastrophe for computations that actually worked fine in practice. James H. Wilkinson, working on early computers at the UK's National Physical Laboratory, flipped the question around and the field was reborn. His idea: do not track how errors accumulate in the answer; instead, account for all the rounding by showing the computed result is the EXACT answer to a slightly perturbed input.
Backward error analysis is the technique of bounding an algorithm's backward error. You prove a statement of the form: the computed y-hat equals f(x + delta-x) exactly, for some delta-x whose relative size is at most a modest multiple of the unit roundoff. The genius is that this neatly separates two concerns that the old forward analysis hopelessly entangled — the quality of the ALGORITHM (captured by the backward error) and the difficulty of the PROBLEM (captured by the condition number). Multiply them and you get the forward error; analyse them apart and each becomes tractable.
This reframing is one of the most important conceptual advances in scientific computing, and it is why Wilkinson won the Turing Award in 1970. It explained why Gaussian elimination, long feared to be hopelessly inaccurate by naive forward bounds, is in fact reliable: it is backward stable. The lesson endures — when judging a numerical method, prove a backward error bound, then read off accuracy by combining it with conditioning. Forward bounds alone tend to be pessimistic and uninformative.
Wilkinson showed that floating-point inner-product accumulation of n terms has a backward error bounded by about n times the unit roundoff: the computed sum is the exact sum of slightly perturbed inputs. This single result underpins the stability proofs of matrix factorizations built from inner products.
One backward-error lemma, reused to certify a whole family of algorithms.
Backward error analysis bounds the algorithm, not the answer. To get the forward error you must still multiply by the condition number; a backward-stable method on an ill-conditioned problem is still inaccurate.