The edges of the representable world
By now you know the machine stores only a finite set of floating-point numbers, laid out under the IEEE 754 standard as a sign, a significand, and an exponent. That finiteness has two edges nobody can avoid. There is a largest number it can hold and a smallest positive number it can hold, and once a computation tries to step past either edge, ordinary arithmetic stops working as you expect. The earlier guides looked inside the representable range — rounding, machine epsilon, the 0.1 + 0.2 surprise. This guide walks to the rim and looks over the side.
In double precision the largest finite value is about 1.8e308, and the smallest positive normalized value is about 2.2e-308. Overflow happens when a result exceeds the largest representable magnitude — the standard does not wrap around or error out; it produces a special value, positive or negative infinity. Underflow happens at the other edge, when a result is smaller in magnitude than the smallest representable positive number — there the value collapses toward zero, passing first through a fringe of tiny gradual numbers we will meet in a moment.
Inf and NaN: the special values that keep arithmetic going
IEEE 754 reserves a few bit patterns for values that are not ordinary numbers, and they are a feature, not an accident. Inf (infinity, with a sign) is what overflow produces and what 1.0/0.0 returns. It obeys sensible rules: Inf + 1 is Inf, 1/Inf is 0, and Inf > x for every finite x. NaN — 'Not a Number' — is the result of an operation with no meaningful answer: 0.0/0.0, Inf - Inf, sqrt(-1.0), or 0.0*Inf. These special values let a long computation keep running to the end instead of crashing, so you can inspect the output and trace where things went wrong.
NaN has one famously strange property: it is unequal to everything, including itself. The test x != x is true exactly when x is NaN, and that is the standard, portable way to detect one. This non-reflexivity is deliberate — it stops a NaN from silently masquerading as a valid number in a comparison. The flip side is that NaN is contagious: any arithmetic touching a NaN yields NaN, so a single bad value early in a pipeline can quietly turn a whole array of results into NaN. When that happens, the fix is never to suppress the NaN; it is to find the operation that first manufactured it.
Subnormals: a soft landing into zero
Right at the underflow edge, IEEE 754 does something clever rather than jumping straight to zero. A normal double always has a leading 1 bit in its significand, which fixes its precision but also fixes how small it can get. Below that smallest normal value, the standard allows subnormal (also called denormal) numbers: it drops the implicit leading 1 and lets the significand have leading zeros, filling the gap between the smallest normal number and zero with evenly spaced tiny values. This is gradual underflow — instead of a cliff, there is a ramp.
The payoff is a property you would otherwise lose: if x and y are different floating-point numbers, then x - y is never exactly zero. Without subnormals, two distinct numbers very close together could subtract to a result too small to represent, snapping to zero — and code that then divides by that difference would suddenly hit a spurious division by zero. Gradual underflow keeps tiny differences alive. The cost is twofold: subnormals carry fewer significant bits, so precision degrades smoothly as you sink into them, and on some hardware operations on subnormals run dramatically slower because they fall off the fast path.
Avoiding the edges: scale before you compute
The good news is that most overflow and underflow is avoidable by rescaling the problem so the intermediate quantities stay near 1, where floating point is happiest. The classic example is the Euclidean length sqrt(a^2 + b^2), the 2-norm of a vector — written naively it overflows for large inputs and underflows for tiny ones, even when the answer is fine. The cure is to factor out the largest magnitude first, so every squared term is at most 1 and the sum cannot overflow or vanish.
Concretely, set m = max(|a|, |b|); if m is zero the length is zero, and otherwise let s = min(|a|, |b|) / m, which always lies in [0, 1] so s*s is harmless. Then the length is m * sqrt(1 + s*s), where the square root sees only a number between 1 and sqrt(2). The big magnitude has been pulled outside as a single multiply, so a^2 can no longer overflow and the inner sum can no longer underflow to zero. The result is identical mathematics, evaluated through a path the finite machine can always represent.
The same idea recurs everywhere in scientific computing. Probabilities are multiplied as sums of logarithms (the log-sum-exp trick subtracts the maximum before exponentiating) so products of many small numbers do not underflow to zero. Linear-algebra routines in double precision balance and scale a matrix before factoring it. The honest framing connects to earlier guides: scaling is a stability technique, the same family of ideas as the stable quadratic formula and cancellation avoidance. You are not changing the mathematics; you are rearranging it so the finite machine never has to represent something it cannot.
Accurate summation: from naive to Kahan
We close with the single most useful accuracy tool in everyday numerical work, building straight on the fact you met earlier: floating-point addition is not associative, so summing a long list of numbers accumulates one rounding error per addition. Add n terms naively and, in the worst case, the total error grows like n times the unit roundoff u — for a million terms that can eat several digits. Kahan summation (compensated summation) fixes this for pennies, by carrying a second variable that remembers the low-order bits each addition throws away and feeds them back into the next step.
function kahan_sum(x[1..n]):
s = 0.0 # running sum
c = 0.0 # compensation: low-order bits lost so far
for i = 1 to n:
y = x[i] - c # add back what we lost last time
t = s + y # this rounding may drop low bits of y
c = (t - s) - y # recover EXACTLY those dropped bits
s = t
return s- y = x[i] - c: pull this term down by the error we carried over from the previous step, so nothing accumulated is lost.
- t = s + y: do the real addition. Because s may be much larger than y, this rounding can silently drop the lowest bits of y.
- c = (t - s) - y: algebraically zero, but NOT in floating point — (t - s) recovers the part of y that survived, and subtracting y leaves exactly the bits that were dropped.
- s = t: commit the rounded sum, and loop. The compensation c now carries the lost bits forward, ready to rejoin the total next time.
Kahan summation cuts the worst-case error from about n*u down to roughly a small constant times u — almost independent of how many terms you add. Two honest caveats keep it real. First, an aggressive optimizing compiler may 'simplify' the line c = (t - s) - y to zero and silently destroy the method; you sometimes need specific compiler flags or volatile variables to forbid that algebraically-correct-but-wrong rewrite. Second, no summation scheme can rescue cancellation when the true answer is a tiny difference of huge terms — that is bad conditioning in the input, and a backward-stable algorithm (one that gives the exact answer to a nearby problem) cannot undo an ill-conditioned problem. Summation accuracy and problem conditioning are different battles.
What this rung leaves you with
Step back over the whole floating-point rung and a single creed emerges: the machine does exact, well-defined arithmetic on a finite set of numbers, and every numerical answer you compute is approximate. You now know its inside (rounding, machine epsilon, non-associative addition) and its edges (overflow to Inf, underflow through subnormals, NaN for the undefined). The interesting question is never 'is it exact?' but 'how large is the error, and can I bound it?' — and you have concrete tools to answer it: scale to dodge the edges, rearrange to dodge cancellation, and compensate to sum accurately.
The next rung lifts this from single operations to whole problems. There you will meet conditioning — how sensitive an answer is to small input changes — and the clean equation that organizes all of numerical analysis: accuracy equals conditioning times stability. You will see why a stable algorithm still fails on an ill-conditioned problem, and why a condition number near 10^8 quietly costs about 8 of your roughly 16 double-precision digits. Everything you just learned about rounding becomes the raw material for reasoning about error in the large.