JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

Quantization, Resolution, and ENOB

Guides 1 and 2 chopped up the time axis: sample fast enough, filter first, and no aliasing. This guide chops up the other axis — voltage — into a finite ladder of numbers. Meet the LSB, the unavoidable quantization noise, the tidy 6.02n + 1.76 dB rule for SNR, and ENOB: the one number that tells you how many of a converter's advertised bits are actually real.

The other axis: chopping up the voltage

The last two guides were entirely about time. Sampling slices the signal into snapshots, the Nyquist theorem tells you to take more than two snapshots per cycle of the fastest thing present, and an anti-alias filter enforces that promise before any high frequency can fold down and masquerade as a low one (aliasing). But a snapshot taken at an instant is still an analog voltage — some real number like 2.71828… volts. A computer cannot store a real number. It must round that voltage to one of a finite list of values. That second step, chopping the voltage axis into a ladder of allowed levels, is quantization, and it is the whole subject of this guide.

Picture the transfer curve of an analog-to-digital converter (ADC). A perfectly straight diagonal line would map every input voltage to its own exact output number. The real thing is a staircase: a band of input voltages all map to the same output code, then the input climbs a little more and the code jumps up one step. The converter is forever rounding the smooth incoming voltage to the nearest rung of a fixed ladder. The art of this guide is understanding how tall those rungs are, what the rounding costs you, and how to tell the honest count of rungs from the marketing one.

Resolution: how fine are the steps (the LSB)

How many rungs does the ladder have? That is set by the number of bits, n. With n bits a converter has exactly 2^n distinct output codes — 8 bits gives 256 levels, 12 bits gives 4096, 16 bits gives 65536. But codes are just numbers; to turn them into volts you need to know how tall the whole ladder is. That total height, the full-scale range (FSR), is anchored by a voltage reference — a stable voltage the converter trusts as its 'top of scale'. Resolution in bits and full scale in volts are two different things, and you must pin both down before any code means a voltage.

The height of one single rung is the least significant bit, the LSB: LSB = FSR / 2^n. This is the finest voltage difference the converter can possibly notice. Make it concrete. Take a converter running off a 5 V reference, so FSR = 5 V. At 12 bits, LSB = 5 V / 4096 = 1.22 mV — change the input by less than that and the output code does not budge. Drop to 8 bits and the rungs balloon to 5 V / 256 = 19.5 mV; climb to 16 bits and they shrink to 5 V / 65536 = 76 uV. Each extra bit cuts the step size in half — that is what 'more resolution' literally buys.

  bits   levels (2^n)   LSB at 5 V FSR    ideal SNR  (6.02n + 1.76 dB)
  ----   ------------   --------------    -----------------------------
    8        256          19.5 mV            ~50 dB
   10       1024           4.88 mV           ~62 dB
   12       4096           1.22 mV           ~74 dB
   14      16384            305 uV           ~86 dB
   16      65536             76 uV           ~98 dB

   one extra bit  =  half the LSB  =  +6.02 dB of SNR
Bits versus everything they decide. Each row halves the step size of the row above and adds about 6 dB of signal-to-noise ratio. The rightmost column is the BEST CASE, derived in the SNR section — real parts fall short of it.

Because the LSB rides on the reference, the reference's quality lands directly on every reading. If your 5 V reference quietly drifts to 5.05 V with temperature, the whole ladder stretches 1 percent and every converted voltage is 1 percent wrong — no number of bits rescues that. This is why precise systems lean on a low-drift bandgap reference rather than a raw supply rail, and why a clever trick called ratiometric measurement deliberately powers the sensor from the same reference so any drift cancels in the ratio. Resolution is meaningless without a trustworthy full scale to measure it against.

Quantization noise: the price of rounding

Rounding to the nearest rung always leaves a leftover. Since the true voltage can sit anywhere within a step, the rounding error is at most half a step in either direction: it lives somewhere between minus LSB/2 and plus LSB/2. For a real, busy signal that wanders all over the ladder, that error hops around unpredictably from sample to sample, and the cleanest way to treat it is as if the converter had added a faint hiss to your signal — quantization noise. It is not a flaw in a particular chip; it is the unavoidable consequence of representing a continuous voltage with a finite number of codes.

How loud is that hiss? If the error is equally likely to be anywhere in its one-LSB-wide window, a standard result for a uniform spread says its RMS value is the window width divided by the square root of twelve: quantization noise RMS = LSB / sqrt(12), which is about 0.29 LSB. (The sqrt(12) is just the spread of a flat distribution — the same math that gives a die's standard deviation.) On our 12-bit, 5 V converter with LSB = 1.22 mV, that is 1.22 mV / 3.46 = 0.35 mV of irreducible noise sitting under everything you measure. You cannot filter it away in the digital domain; it was baked in the instant you rounded.

SNR and the magic 6.02n + 1.76

Now we can compare the signal against that hiss and get the converter's headline number, its signal-to-noise ratio (SNR). Use the kindest test signal — a full-scale sine wave that swings across the entire ladder. Its peak is FSR/2, so its RMS is (FSR/2) / sqrt(2). Divide that by the quantization-noise RMS of LSB / sqrt(12), substitute LSB = FSR / 2^n, and the FSR cancels cleanly, leaving SNR = 2^n times sqrt(1.5). Expressed in decibels this collapses to one famous line: SNR = 6.02 times n + 1.76 dB.

That tidy formula is worth memorizing, because it turns bits into decibels at a glance. Each extra bit adds 6.02 dB — exactly the 'half the LSB' from before, since halving the noise is +6 dB. A 12-bit converter tops out at 6.02 times 12 + 1.76 = 74 dB; a 16-bit at 6.02 times 16 + 1.76 = 98 dB; a humble 8-bit at about 50 dB. But read the assumptions out loud: this is an ideal converter measured with a full-scale sine. Feed it a signal that only uses half the range and you immediately throw away 6 dB — one whole bit of effective performance — because the hiss is fixed while your signal got smaller. Headroom you don't use is resolution you don't get.

  1. Find the smallest change you must actually resolve. Say a sensor spans 0 to 5 V (so FSR = 5 V) and the spec demands you distinguish steps of 1 mV.
  2. Demand an LSB no bigger than that target: LSB = FSR / 2^n must be <= 1 mV, so 2^n >= 5 V / 1 mV = 5000. Since 2^12 = 4096 is too few and 2^13 = 8192 is enough, you need at least 13 bits.
  3. Add margin. You want the wanted detail sitting several LSBs above the floor, not balanced on the very last rung, so reach for a 14- or 16-bit part rather than scraping by at exactly 13.
  4. Reality-check against the front end. If the amplifier ahead of the converter already adds 2 mV of its own noise, a 13-bit (0.6 mV LSB) part merely measures that noise more finely. Fix the analog side first — there is no point buying an LSB smaller than your noise floor.

ENOB: the honest bit count

That last step exposes the gap this whole guide has been circling toward. The 6.02n + 1.76 line is a ceiling no real part touches, because quantization noise is never the only noise. The converter's own thermal noise, the timing jitter on the sample clock from guide 1, harmonic distortion when the staircase bends away from a straight line, and static errors in the rung heights themselves — captured as INL and DNL — all pile on top. Add them all and you get the measured SINAD (signal-to-noise-and-distortion). The single most honest spec of any converter is what bit count that real SINAD corresponds to.

Invert the magic formula and you get exactly that: the effective number of bits (ENOB) = (SINAD - 1.76) / 6.02. It asks a blunt question — 'an ideal converter of how many bits would have this much total noise?' Take a chip the marketing calls 16-bit whose datasheet, read honestly, lists SINAD = 84 dB. Then ENOB = (84 - 1.76) / 6.02 = 13.7 bits. You paid for 16 and you got the real-world performance of a flawless 13.7-bit part; the bottom two-plus bits are buried in noise and distortion. ENOB, not the nameplate, is the number you design around.

There is one lever that genuinely buys resolution back: oversampling. Quantization noise power is roughly fixed, but it spreads evenly across the whole band up to half the sample rate. Sample far faster than you need, then digitally low-pass and decimate, and you keep all the signal while discarding most of the noise band — every 4x in sample rate wins about 6 dB, one extra bit. Bolt on noise shaping, which pushes quantization noise up out of the band of interest, and the payoff per octave is far larger; that pairing is exactly how a sigma-delta converter trades blistering speed for many quiet bits, and it is where guide 5 is headed.