Floating-Point Arithmetic & Number Representation

overflow and underflow

Every floating-point format can name only numbers within a bounded magnitude range. Push a result past the largest representable value and you OVERFLOW; squeeze it below the smallest representable nonzero value and you UNDERFLOW. Think of a car odometer that maxes out, or a scale too coarse to register a feather: the number you wanted simply has no home in the format.

Overflow happens when a result exceeds the largest finite number (about 1.8 * 10^308 in double). Under IEEE 754 it does not crash; the result becomes +Inf or -Inf and an overflow flag is raised, and that infinity then propagates through the rest of the computation. Underflow happens when a nonzero result is smaller in magnitude than the smallest normal number (about 2.2 * 10^(-308)). Thanks to subnormals the value degrades GRADUALLY — first into reduced-precision subnormals, and only finally to 0 — rather than snapping straight to zero, which is what 'gradual underflow' means.

Both are usually symptoms of a formula written without regard to scale. The classic example is the Euclidean length sqrt(x^2 + y^2): if x = 1e200, then x^2 = 1e400 overflows to Inf and the result is Inf even though the true answer is a perfectly representable 1e200. The cure is to rescale: factor out the largest magnitude, m = max(|x|,|y|), and compute m * sqrt((x/m)^2 + (y/m)^2) — exactly what a good library hypot function does. Underflow can quietly flush a needed small quantity (a probability, a residual) to 0; the remedies are to work in log-space, to rescale, or to reorder operations so intermediate values stay in range.

Computing exp(1000.0) in double overflows to +Inf (the true value ~10^434 exceeds 1.8e308), and exp(-1000.0) underflows toward 0. In statistics this bites when multiplying many probabilities: a product of thousands of values near 0.01 underflows to 0, so practitioners sum log-probabilities instead and exponentiate only at the end.

Rescale or work in log-space to keep values inside the format's range.

Overflow to Inf is loud and often noticed; underflow to 0 is silent and more insidious, because the computation keeps running with a wrong, plausible-looking small result. Watch small products and exponentials especially.

Also called
overflowunderflow溢位下溢