the law of the iterated logarithm
The strong law says the sample mean converges to mu; the central limit theorem says the fluctuations around mu are of size sigma/sqrt(n). But these leave a gap: how big do the EXTREME fluctuations get as you watch a single infinite run forever? Sometimes the average will wander unusually far from mu — how far, exactly, and how often? The law of the iterated logarithm is the surprisingly precise fine print that fills this gap.
Precisely, for iid variables with mean 0 and variance sigma^2, look at the partial sums S_n. The law says: with probability 1, the ratio S_n / sqrt(2 * sigma^2 * n * ln(ln n)) has limsup equal to +1 and liminf equal to -1. Decode that. The 'envelope' that the sums repeatedly brush against is sqrt(2 * sigma^2 * n * ln(ln n)) — a touch larger than the CLT's sqrt(n) scale, inflated by the doubly-iterated logarithm ln(ln n) (which grows agonizingly slowly). Limsup = +1 means the sums climb up and kiss the upper envelope infinitely often but never permanently exceed it; liminf = -1 says the same on the downside. The path forever oscillates, scraping a precisely calibrated boundary.
It is the bridge between the two great laws. The CLT describes the typical size of S_n at a single n (order sqrt(n)); the LIL describes the size of the rare RECORD excursions across all n (order sqrt(n ln ln n)). The extra ln(ln n) factor is exactly enough to be crossed infinitely often but with vanishing frequency. The honest nuance: limsup and liminf are about events that recur forever, not about S_n settling — the partial sums never converge (they grow), only the running average does. The LIL is the sharpest honest statement about just how erratic that random walk of partial sums remains, no matter how long you watch.
For a fair coin counting heads minus tails as S_n, the LIL says S_n repeatedly reaches up to about sqrt(2 n ln ln n) and down to about -sqrt(2 n ln ln n), infinitely often, but never permanently breaks past. At n = a billion, ln(ln n) ≈ 3.0 — so even after a billion flips the doubly-logarithmic correction is only a factor near sqrt(6).
Partial sums repeatedly graze an envelope of size sqrt(2 sigma^2 n ln ln n) — bigger than sqrt(n), but only by a crawling ln ln n.
The LIL is about the limsup and liminf of fluctuations recurring forever, not about the sums settling — partial sums never converge, only the average does.