Bennett's inequality
/ BEN-it /
Bennett's inequality is the sharpest of the classical variance-and-bound concentration inequalities for sums of independent bounded variables. Where Hoeffding uses only the range and Bernstein uses the variance plus a relaxed rate, Bennett extracts the exact best-possible exponent from the same hypotheses (a variance bound and an almost-sure upper bound on each summand). It is the inequality whose rate function correctly interpolates between the Gaussian centre and the Poisson tail, and Bernstein is just a convenient lower bound on Bennett's rate.
The statement: let X_1, ..., X_n be independent, centered, with X_i <= b almost surely and sum Var(X_i) <= sigma^2. Then for t > 0, P(sum X_i >= t) <= exp(-(sigma^2 / b^2) h(b t / sigma^2)), where h(u) = (1 + u) log(1 + u) - u is the Poisson rate function. The function h is the key character: it is the rate function of a centered Poisson variable, h(u) ~ u^2/2 for small u (recovering the Gaussian tail exp(-t^2 / 2 sigma^2)) and h(u) ~ u log u for large u (recovering a tail like (sigma^2/(b t))^(t/b) e^(t/b), faster than any pure exponential). The reason the Poisson rate function appears is structural: among all distributions with a given variance and a given upper bound b, the extremal large-deviation behaviour is achieved by a (compound-)Poisson-type distribution, so Bennett is exactly capturing that worst case. Bernstein's inequality follows from Bennett by the elementary bound h(u) >= u^2 / (2(1 + u/3)), which trades the exact Poisson rate for the algebraically simpler two-regime form.
In practice Bennett is used where the extreme tail genuinely matters — rare-event estimation, the analysis of estimators driven by occasional large summands, and any setting where one needs the true log-rate rather than a convenient bound. Its caveat is the mirror image of its strength: it requires both a variance control and an almost-sure one-sided bound b, and it is one-sided (a symmetric two-sided statement requires applying it to X and to -X with the relevant bounds). When b -> 0 with sigma^2 fixed (many tiny summands), Bennett's bound converges to the exact Gaussian tail exp(-t^2 / 2 sigma^2), which is the central limit theorem made quantitative; when the summands are not bounded above, Bennett does not apply and one needs sub-exponential (Bernstein) or heavier-tail machinery.
Sum n indicators of independent rare events each with probability p, so each centered summand is at most b = 1 and has variance p(1-p) ~ p; sigma^2 = np. Bennett gives P(sum >= np + t) <= exp(-np h(t/(np))), which for the extreme tail t = np reproduces the genuine Poisson upper tail e^(-np) (np)^(2np) / (2np)! flavor that Bernstein only bounds crudely.
Bennett's rate is the Poisson rate function h(u) = (1+u)log(1+u) - u.
Bennett requires a one-sided almost-sure bound X_i <= b and is itself one-sided; it is strictly sharper than Bernstein, which follows by the elementary inequality h(u) >= u^2/(2(1+u/3)). The Gaussian limit b -> 0 makes it a quantitative CLT.