reduced-precision arithmetic
When you estimate a tip, you do not work out 18.000000% to ten decimal places — you round to a couple of dollars, because that is precise enough for the purpose and far quicker. Reduced-precision arithmetic is hardware doing the same thing: representing numbers with fewer bits than the usual 32- or 64-bit formats, accepting a little rounding error in exchange for big gains in speed, energy, and memory — wherever the task can tolerate it.
Concretely, ordinary computation often uses 32-bit floats (about 7 decimal digits of precision) or 64-bit doubles. Reduced precision uses narrower formats: bfloat16, a 16-bit float that keeps the full exponent range of a 32-bit float but throws away precision in the mantissa (so it represents roughly the same range of magnitudes but coarser detail); or int8, an 8-bit integer with only 256 possible values, used after 'quantizing' a network's weights to that scale. Narrower numbers mean each value takes less memory and less bandwidth to move, and the multiply-add hardware is smaller — so you can fit far more arithmetic units in the same chip and run them at lower energy. A multiply on 8 bits costs a tiny fraction of a 32-bit one.
The honest point is that this is a domain-specific trade, not a universal free win. It works for deep learning because neural networks are statistically robust — they are trained to tolerate noise, and a little rounding rarely changes the final answer (a cat is still classified as a cat). It would be reckless for, say, a bank ledger or a physics simulation where small errors accumulate into wrong results. Choosing reduced precision means knowing exactly how much error your domain can absorb; bfloat16 was designed for machine learning precisely because the field tolerates low mantissa precision but needs the wide exponent range to avoid overflow during training.
Quantizing a trained neural network from 32-bit floats to int8 typically shrinks the model to a quarter of its size and lets it run several times faster on an accelerator, while accuracy drops by only a fraction of a percent — a great trade. Trying the same int8 quantization on a scientific simulation could turn small rounding errors into a completely wrong result.
Fewer bits per number: smaller, faster, lower-energy — but only where the domain tolerates the rounding.
bfloat16 is not the same as the standard IEEE 16-bit float (FP16): bfloat16 keeps a wider exponent (same range as FP32) at the cost of fewer mantissa bits, which suits training. Reduced precision is safe only when you have measured that your domain can absorb the error.