the floating-point model

Before reasoning about errors, numerical analysts agree on a clean mathematical picture of what a computer actually stores. The floating-point model is that picture: every machine number has the shape value = ±d.ddd...d * base^e, a sign, a string of significant digits (the significand), and an exponent that slides the radix point. In base 2 with a fixed number of significand bits and a bounded exponent, this describes a finite, discrete set F of representable numbers.

Two facts about F matter most. First, F is FINITE: there are a largest and a smallest positive value, and only finitely many numbers between them, so most real numbers must be rounded to the nearest member of F. Second, F is UNEVENLY spaced: near 1 the gaps are tiny, near a million they are larger, and near a billionth they are tinier still, because each step is about base^e and the exponent grows with magnitude. The grid stretches logarithmically, giving roughly constant RELATIVE spacing rather than constant absolute spacing.

The model's payoff is the standard model of a rounded operation: rounding any real x to the nearest machine number, written fl(x), satisfies fl(x) = x(1 + delta) with |delta| <= u, and likewise fl(x op y) = (x op y)(1 + delta). This one inequality, applied operation by operation, is the foundation of all rounding-error analysis: it lets you bound the error of an entire algorithm without tracking individual bits, and it is what makes statements like 'this method is backward stable' provable rather than hopeful.