the failure of the CLT without finite variance
The central limit theorem feels almost magical — averages always become normal. But that magic has a price written into the hypotheses: a FINITE variance. When the tails of a distribution are so heavy that the variance is infinite (or even the mean does not exist), the spell breaks completely. The Cauchy distribution is the textbook cautionary tale, and it fails in the most dramatic way imaginable.
The Cauchy distribution has the gentle bell-like density 1/(pi(1 + x^2)), symmetric and innocent-looking. Yet its tails decay so slowly that the integral defining the mean does not converge, and the variance is infinite. Here is the shocking consequence: if X_1, ..., X_n are iid Cauchy, their sample mean X-bar_n is itself EXACTLY Cauchy with the same scale — averaging does nothing. Take ten thousand draws and average them, and the average is no more concentrated than a single draw. There is no sqrt(n) shrinking, no march toward a narrow normal; the distribution of the average simply stands still.
The lesson is honest and important: the CLT is a theorem with hypotheses, not a law of nature. Finite variance is the gatekeeper, and heavy-tailed data (financial returns in crises, sizes of cities, network traffic) can violate it, where averages stay wild and 'the average of many is reliable' is false. Such variables instead fall under stable distributions, a different limit family. Whenever someone invokes the CLT, the right first question is: does this quantity even have a finite variance? If not, the bell may never come.
Average n = 10000 independent standard Cauchy draws. You expect the average to settle near a centre, as with a normal — but it does not: X-bar_10000 is again standard Cauchy, with the same fat tails and the same chance of a wild value as a single draw. The sample mean of Cauchy data is no estimator at all.
Averaging iid Cauchy variables yields the SAME Cauchy: no concentration, no normal limit — the CLT's finite-variance hypothesis is essential.
The CLT is a theorem with hypotheses, not a law of nature: it requires finite variance. The Cauchy has neither finite variance nor a mean, and its sample mean stays Cauchy — averaging buys nothing.