Why integers run out of room
Everything so far in this rung — the ALU doing addition and subtraction, the shift-and-add multiplier, the slow divider — worked on integers. Integers are wonderful: every value is exact, and the answer is either right or it overflows, with nothing fuzzy in between. But a fixed-width integer is a ruler with evenly spaced marks and a hard edge. With 32 bits you get whole numbers up to about four billion and not one more, and you cannot represent one-half at all.
Science needs both the mass of a star (about 2 times 10^30 kilograms) and the mass of an electron (about 9 times 10^-31 kilograms) in the same program, and no integer ruler can hold both. The fix is to stop spacing the marks evenly. Instead of fixed marks, write each number in a kind of binary scientific notation — a fraction times a power of two — so the spacing of representable values stretches as numbers grow large and crowds together as they shrink toward zero. That is exactly what floating point does: the binary point floats to wherever the value needs it.
The IEEE 754 format: sign, exponent, significand
Almost every machine agrees on one layout, the IEEE 754 standard, which is why a number saved on your laptop reads back the same on a phone or a supercomputer. A 32-bit single-precision float splits its bits into three fields: 1 sign bit, 8 exponent bits, and 23 fraction bits. The value is sign times 1.fraction times 2^(exponent minus 127). The leading 1 before the binary point is not stored — it is always there for normal numbers, so making it implicit buys one free bit of precision. The 23 stored fraction bits plus that hidden 1 form the 24-bit significand.
32-bit IEEE 754 single precision
[ s ][ exponent ][ fraction ]
1 8 23 = 32 bits
value = (-1)^s x 1.fraction(binary) x 2^(exponent - 127)
example: encode -6.5
6.5 = 110.1 (binary) = 1.101 x 2^2
s = 1 (negative)
exponent = 2 + 127 = 129 = 10000001
fraction = 101 0000... (the bits after the implicit 1.)
-> 1 10000001 10100000000000000000000The exponent uses a bias of 127 instead of two's complement so that floats compare in the same order as integers, a handy trick for hardware. The standard reserves the two extreme exponent patterns for special meanings: all-zeros marks zero and tiny subnormal numbers, and all-ones marks infinity (when the fraction is zero) and Not-a-Number (when it is not). NaN is the honest result of 0/0 or the square root of a negative — a value that says 'this question has no real answer' and then poisons every later computation it touches, so an error never quietly looks like a normal number.
Inside the floating-point unit: how an addition works
Adding two floats is not as simple as adding their bits, because two numbers with different exponents have their binary points in different places. You cannot add 1.5 times 2^3 to 1.0 times 2^0 by stacking the significands — first you must line up the points, just as adding 12.0 and 0.34 in decimal means writing them so the decimal points sit under each other. The floating-point unit (FPU) is a small specialised datapath built to do this alignment, the add, and the cleanup in sequence.
- Align exponents: shift the significand of the smaller-exponent number right until both share the larger exponent. To add 1.5 x 2^3 and 1.0 x 2^0, shift the second right by 3 bits so it becomes 0.001 x 2^3.
- Add the significands using essentially the same integer adder from earlier guides — the carry-lookahead adder you already met handles this step.
- Normalize the sum: shift it so exactly one non-zero digit sits before the binary point (the form 1.xxxx), adjusting the exponent to match. A sum like 10.1 x 2^3 becomes 1.01 x 2^4.
- Round the result back to the available number of significand bits, then check for overflow (exponent too large, becomes infinity) or underflow (too small, becomes subnormal or zero).
That alignment step is where precision quietly leaks away. When you shift the smaller number right to match exponents, its lowest bits fall off the end of the significand. If the two numbers differ enormously in size — adding 1.0 to 2^30 — every bit of the small one is shifted into oblivion and the answer is simply the large number unchanged. This is not a bug; it is the price of a fixed-width significand, and it is why summing a long list of numbers from largest to smallest can lose terms that summing smallest-first would keep.
Rounding: guard, round, and sticky bits
After the add and normalize, the true result often has more significant bits than the format can store, so the hardware must round. The naive idea — just chop off the extra bits — is biased, because it always rounds toward zero and the tiny errors all lean the same way, accumulating over millions of operations. The standard's default is smarter: round to nearest, ties to even. Pick the closest representable value; if the true value sits exactly halfway between two, choose the one whose last bit is 0. Ties-to-even has no directional bias, so errors cancel rather than pile up.
But here is a subtle hardware problem: to round correctly the FPU must know what was in the bits it is about to discard, yet it has already thrown most of them away during alignment. The elegant fix is to keep just three extra bits past the significand: the guard, round, and sticky bits. The guard and round bits are the first two discarded bits, kept verbatim. The sticky bit is the logical OR of every bit below them — it remembers whether anything non-zero was shifted out, which is exactly what you need to break a tie correctly.
Round-to-nearest-even is just the default; the rounding mode is selectable. The standard also offers round toward zero, toward positive infinity, and toward negative infinity. Those directed modes power interval arithmetic, where you compute a result twice — once rounding up, once rounding down — to get a guaranteed range that brackets the true mathematical answer. The whole point of pinning rounding down to the bit is reproducibility: the same program on conforming hardware gives bit-identical results everywhere, which would be impossible if each chip rounded however it liked.
The honest surprises of floating point
The most famous surprise: 0.1 + 0.2 does not equal 0.3. The reason is not a hardware bug but a representation limit. Just as 1/3 has no exact finite decimal (0.3333...), the value 0.1 has no exact finite binary fraction — it repeats forever in base two. So the moment you write 0.1, the machine already stores the nearest representable value, which is slightly off, and the small errors in 0.1 and 0.2 do not cancel to exactly 0.3. The arithmetic is correct to the last bit; the inputs were never exactly what you typed.
Floating point also breaks rules you trust from algebra. Addition is not associative: (a + b) + c can differ from a + (b + c), because rounding happens after each step and a different grouping rounds at different moments. Catastrophic cancellation is nastier — subtracting two nearly equal numbers wipes out their leading matching digits and promotes rounding noise from the bottom into the leading digits of the answer, so a result that looks fine can be almost entirely error. Numerical libraries are full of cleverly rearranged formulas written precisely to dodge these traps.
None of this makes floating point untrustworthy — it makes it a precise tool with documented limits. The IEEE 754 standard is a triumph of honest engineering: it does not pretend real numbers fit in 32 or 64 bits, it specifies exactly how they are approximated, rounds to the nearest value to the last bit, signals trouble with infinity and NaN instead of hiding it, and guarantees the same answer on every conforming machine. Understanding floating point, like understanding a cache or a pipeline, means knowing not just that it works but precisely where and why it bends.