the Orlicz norm
/ OR-litch /
The Orlicz norm is the device that turns the qualitative classes 'sub-Gaussian' and 'sub-exponential' into honest norms — numbers you can add, scale, and compare. Once you have a norm, the sub-Gaussian variables form a Banach space, sums are controlled by triangle inequalities, and a hundred concentration arguments become routine bookkeeping. The two norms that matter for concentration are the psi_2 (sub-Gaussian) norm and the psi_1 (sub-exponential) norm, built from the Young functions psi_2(x) = e^(x^2) - 1 and psi_1(x) = e^x - 1.
For a Young function psi (convex, increasing, psi(0)=0), the Orlicz norm of X is ||X||_psi = inf{ t > 0 : E[psi(|X|/t)] <= 1 }. The infimum picks the smallest scale t at which the random variable, rescaled, has expected Orlicz cost at most 1. Concretely ||X||_psi2 = inf{ t > 0 : E[exp(X^2/t^2)] <= 2 } is the sub-Gaussian norm, and ||X||_psi1 = inf{ t > 0 : E[exp(|X|/t)] <= 2 } is the sub-exponential norm. A variable is sub-Gaussian iff ||X||_psi2 < infinity, and sub-exponential iff ||X||_psi1 < infinity; these finite norms are equivalent (up to absolute constants) to the moment-growth and tail constants from the respective definitions. The fundamental link between them is ||XY||_psi1 <= ||X||_psi2 ||Y||_psi2 and, in particular, ||X^2||_psi1 = ||X||_psi2^2, which is the rigorous form of 'square of sub-Gaussian is sub-exponential'.
Why a norm rather than a constant? Because it makes operations clean. The triangle inequality lets you split errors; centering costs only a constant, ||X - E[X]||_psi <= 2 ||X||_psi; and for independent centered sub-Gaussians the key additivity is ||sum X_i||_psi2^2 <= C sum ||X_i||_psi2^2, the variance-proxy addition rule that produces Hoeffding's bound. The norm is unique up to the absolute constant hidden in the choice of '2' on the right side of E[psi(...)] <= 2 (some texts use 1); this is the same harmless constant-ambiguity that pervades the whole subject. Orlicz norms also generalize the L^p scale: ||X||_psi controls all L^p norms simultaneously, which is exactly the moment-growth characterization.
For X ~ N(0, sigma^2), E[exp(X^2/t^2)] = (1 - 2 sigma^2/t^2)^(-1/2), which equals 2 when t^2 = 8 sigma^2 / 3, so ||X||_psi2 = sigma sqrt(8/3) ~ 1.63 sigma — finite and proportional to sigma, as a sub-Gaussian norm should be. A Rademacher sign has ||X||_psi2 = 1/sqrt(log 2).
The psi_2 norm of a Gaussian is a fixed multiple of its standard deviation.
The exact numerical value of an Orlicz norm depends on the convention (the '2' vs '1' on the right, and whether one uses psi(x)=e^(x^2)-1 or e^(x^2)); only the equivalence class up to absolute constants is canonical. Do not compare psi norms across textbooks expecting identical numbers.