Computer Arithmetic

normalization in floating-point arithmetic

In ordinary scientific notation we always write a single non-zero digit before the decimal point: we say 3.0 times 10^2, not 0.3 times 10^3 or 30 times 10^1, even though all three are equal. Insisting on that one canonical form is normalization. Floating point does the same in binary: a normalized number has exactly one 1 just before the binary point, written 1.fff... times 2^e. Keeping every number in this standard shape means each value has a single representation and every available significand bit carries real precision instead of being wasted on leading zeros.

After an arithmetic step the result is usually not normalized, so the FPU fixes it. Two cases. If adding two close numbers cancels the high bits — say the result came out as 0.0011 times 2^e — the hardware shifts the significand left until the leading 1 reaches the standard position and decreases the exponent by the number of shifts (here left 3, so exponent becomes e minus 3). If a multiply or add produced a carry beyond the leading 1 — say 11.01 times 2^e — it shifts right by one and increases the exponent by one. A neat bonus of always having a leading 1 is that, since it is always there, IEEE-754 does not store it: it is the 'hidden' or implicit bit, buying one extra bit of precision for free.

Normalization is where two of floating point's hazards appear. Massive cancellation — subtracting two nearly equal numbers — can leave only a few meaningful bits, which then get shifted up and padded, so the answer looks precise but is mostly noise (loss of significance). And if normalization pushes the exponent below the smallest representable value, the number underflows toward zero; IEEE-754 softens the cliff with gradual underflow using subnormal numbers, which are not normalized and trade away precision to represent tiny magnitudes.

Adding significands gives 0.00101 x 2^4. Normalize by shifting left 3 so the leading 1 sits before the point: 1.01 x 2^1 (exponent 4 minus 3). If instead a sum overflowed to 10.1 x 2^4, shift right 1 to get 1.01 x 2^5.

Shift the significand and adjust the exponent until exactly one 1 sits before the binary point.

Subtracting nearly equal numbers (catastrophic cancellation) leaves few real bits; normalization then shifts in zeros, so the result looks precise but is mostly noise. The fix is in how you write the math, not the hardware.

Also called
normalize正規化規格化