forward error

Forward error is the plain, intuitive measure of being wrong: the distance between the computed answer x_hat and the true answer x. As a relative quantity it is ||x_hat - x|| / ||x||. It is what you really care about — but, frustratingly, it is usually the hardest quantity to know directly, because computing it would require already knowing x.

The central relationship of numerical analysis ties it to the two things you can estimate: forward error <= condition number times backward error. The backward error you can certify from the residual; the condition number you can estimate from the matrix. Their product bounds how far your computed answer can be from the truth.

This factorization is illuminating. A backward-stable algorithm keeps the backward error near machine precision, so the forward error is essentially the condition number times machine precision. If the problem is well-conditioned the answer is accurate; if it is ill-conditioned the same excellent algorithm yields a large forward error through no fault of its own.

So when an answer is inaccurate, the inequality tells you where to look. Large backward error points at an unstable algorithm worth replacing. Small backward error but large forward error points at an ill-conditioned problem worth reformulating. The two diagnoses call for completely different fixes.