a backward-stable algorithm

Imagine hiring a courier. You cannot demand they deliver to the exact GPS point you specified to the millimetre — the world is not that precise. But you can demand they deliver to a point indistinguishable, given the noise in your own address, from where you meant. A backward-stable algorithm is that ideal courier: it returns the EXACT answer to a problem that is only negligibly different from the one you asked. This is the gold standard of numerical computing — and, importantly, the best you can usually hope for.

Formally, an algorithm computing y = f(x) is backward stable if, for every input x, the computed y-hat is the exact value f(x + delta-x) for some small perturbation delta-x with ||delta-x|| / ||x|| of order the unit roundoff (about 10^-16 in double precision). It does not promise the right answer to your problem; it promises the right answer to a problem within rounding distance of yours. Many staple algorithms have this property: Gaussian elimination with partial pivoting, the QR factorization via Householder reflections, and the standard solvers in LAPACK are all backward stable.

Here is the part people miss: backward stability does NOT mean a small forward error. Combined with the master rule, a backward-stable algorithm guarantees forward error of order (condition number times unit roundoff). So on a well-conditioned problem you get a beautifully accurate answer, but on an ill-conditioned problem the same backward-stable algorithm gives a large forward error — and that is not a defect of the algorithm, it is the problem's conditioning showing through. Backward stability is therefore the right thing to demand of a method: it has done everything an algorithm can do.