heavy tails and when the mean misleads

The mean is so familiar that we reach for it reflexively, but there are whole families of situations where it is a poor, even meaningless, summary. The culprit is usually heavy tails: distributions where extreme values, though individually rare, are common and large enough to dominate. In such worlds the 'average' tells you about a typical outcome that almost never happens.

Think of a small town where everyone earns about 40 thousand a year, and then one billionaire moves in. The mean income leaps to millions while the median (the person in the middle) barely moves and still describes nearly everyone. That is the signature of right-skewed, heavy-tailed data: a handful of giant values yank the mean far from the bulk of the distribution. For strongly heavy-tailed laws the mean is unstable — sample averages keep jumping around as you collect more data instead of settling down — and in the extreme case of the Cauchy distribution the mean does not exist at all: the integral that defines E[X] diverges, and the average of n Cauchy samples is no more accurate than a single sample, no matter how large n is. The law of large numbers simply does not apply, and the central limit theorem fails because it requires finite variance.

The practical lesson: choose your summary to fit the shape. For symmetric, light-tailed data the mean is ideal. For skewed or heavy-tailed data — incomes, city sizes, insurance losses, file sizes, financial crashes — the median (and other quantiles) are far more honest, and you should report the tail behaviour explicitly rather than hide it behind one number. The mean is a tool, not an oracle; knowing when it breaks is part of using it well.