the entropy method

The entropy method is a systematic, almost mechanical route to concentration that replaces the ad hoc cleverness of the martingale or Talagrand arguments with a single recurring quantity — the entropy of the exponential e^(lambda f) — and a single structural tool: that this entropy tensorizes over independent coordinates. It is the technology behind the modern, sharpest, variance-aware concentration inequalities (Bousquet's bound for empirical-process suprema, the bounded-differences inequality with variance, self-bounding functions), and it connects concentration directly to functional inequalities like the logarithmic Sobolev inequality.

Define, for a non-negative random variable Y, its entropy Ent(Y) = E[Y log Y] - E[Y] log E[Y] (a non-negative quantity by Jensen). The method studies Ent(e^(lambda f)) as a function of lambda. The first pillar is the Herbst argument: if one can prove a 'modified log-Sobolev' inequality of the form Ent(e^(lambda f)) <= (lambda^2 sigma^2 / 2) E[e^(lambda f)] for all lambda, then dividing through turns this into a differential inequality for the log-MGF H(lambda) = log E[e^(lambda f)], namely (H(lambda)/lambda)' <= sigma^2/2, which integrates to H(lambda) <= lambda E[f] + lambda^2 sigma^2/2 — a sub-Gaussian MGF bound, hence Gaussian concentration by Chernoff. The second pillar is tensorization: for a function of independent coordinates, the entropy of e^(lambda f) is bounded by the sum over coordinates of the conditional entropies obtained by resampling one coordinate at a time, Ent(e^(lambda f)) <= sum_i E[Ent_i(e^(lambda f))]. This sub-additivity is what reduces a high-dimensional problem to a one-coordinate computation; you only have to control how much e^(lambda f) varies when a single input is resampled, and the dimension-free aggregation is automatic.

The method's superpower over McDiarmid and Azuma is that the per-coordinate term naturally involves a VARIANCE-like quantity — the conditional fluctuation when one coordinate is resampled — rather than a worst-case difference, so it produces Bernstein-type and self-bounded inequalities. A self-bounding function (one where sum_i (f - f_i)^2 <= f, with f_i the value after dropping coordinate i) automatically concentrates with a Poissonian rate via this route. The honest caveat is that the method's output is exactly as good as the log-Sobolev-type inequality you can prove for the underlying measure; for product measures the tensorization gives you that inequality for free, but for dependent variables (Markov chains, Gibbs measures) one needs a genuine log-Sobolev or Poincare constant, and obtaining that constant can be the entire difficulty — concentration is then as hard as the spectral gap.