Probability Inequalities & Concentration

a sub-Gaussian random variable

The normal (Gaussian) distribution is the gold standard for 'light tails': its probability of a large deviation dies off like e^(-x^2/2), extraordinarily fast. Many variables that are not exactly normal — any bounded variable, for instance — share this same blazing tail decay. A sub-Gaussian random variable is precisely one whose tails are at least as light as some Gaussian's: a clean, portable label for 'this variable concentrates at least as well as a bell curve'.

There are several equivalent ways to pin it down, but the most useful is via the moment generating function: X (with mean mu) is sub-Gaussian with parameter sigma if E[e^(t(X - mu))] <= e^(sigma^2 t^2 / 2) for all real t. That is exactly the MGF of a Normal(0, sigma^2), so the definition literally says 'no fatter than a Gaussian'. The immediate payoff is that Chernoff's method then yields a Gaussian-shaped tail bound for free: P(|X - mu| >= a) <= 2 e^(-a^2 / (2 sigma^2)). Crucially, sums of independent sub-Gaussians are again sub-Gaussian, with parameters that add in quadrature — so this property passes cleanly through the averaging operations at the heart of concentration.

Sub-Gaussianity is the modern organizing concept that unifies the bounded-variable results (Hoeffding's lemma is exactly the statement that a bounded variable is sub-Gaussian) and lets the whole machinery extend to unbounded-but-light-tailed variables. It is the natural setting for clean high-dimensional probability and learning theory. The honest boundary: many important variables are NOT sub-Gaussian — anything heavy-tailed (the exponential is only sub-exponential; the Cauchy is far worse), where deviations are too likely for a Gaussian bound to hold, and you must use weaker tail bounds or Bernstein-type inequalities instead.

Any variable confined to [a, b] is sub-Gaussian with parameter sigma = (b - a)/2 — this is Hoeffding's lemma. So a single coin-flip outcome in {0, 1} is sub-Gaussian with sigma = 1/2, and averaging n independent flips stays sub-Gaussian with parameter shrinking like 1/sqrt(n), giving the exponential tail behind Hoeffding's inequality.

Tails no heavier than a Gaussian's — so Chernoff hands you an e^(-a^2) tail bound automatically.

Sub-Gaussian is a strong, light-tail condition. Heavy-tailed variables (exponential is only sub-exponential; Cauchy worse) are not sub-Gaussian, and their deviations are too likely for a Gaussian-style bound.

Also called
subgaussian variablesub-Gaussian distribution次高斯變數亞高斯隨機變數