moments and central moments
A distribution is a whole shape, but we often want to summarize it by a handful of numbers — its centre, its spread, its lopsidedness, its peakedness. Moments are a systematic ladder of such summaries, each one a particular average that pins down one more feature of the shape. The word comes from physics, where the same sums describe the centre of mass and the moment of inertia of a body.
The k-th (raw) moment is E[X^k]: the first moment E[X] is the mean. The k-th central moment is E[(X - mu)^k], the moment measured from the centre: the first central moment is always 0 (deviations average out), and the second central moment E[(X - mu)^2] is exactly the variance. Higher central moments capture finer shape — the third measures asymmetry (it feeds skewness) and the fourth measures how heavy the tails and how peaked the centre are (it feeds kurtosis). All of these are computed by LOTUS: feed the function x^k into the original distribution and average.
Moments are the numerical fingerprint of a distribution, and the moment generating function is a single object that produces them all on demand. Two honest caveats. Higher moments demand that the relevant integral converges, and heavy-tailed distributions can have only finitely many moments — beyond a point E[X^k] is infinite or undefined (the Cauchy has no moments at all, not even a mean). And even an infinite list of moments does not always determine the distribution uniquely; in pathological cases two different distributions can share every moment.
For a standard normal (mean 0): the first moment is 0, the second moment (= variance) is 1, the third central moment is 0 (perfectly symmetric, so no skew), and the fourth central moment is 3. These four numbers already say 'centred, unit spread, symmetric, normal-shaped tails'.
Moments are a ladder of averages — mean, variance, skew, kurtosis — each pinning one more feature of the shape.
Heavy-tailed distributions can have only finitely many moments: beyond some order E[X^k] is infinite or undefined. The Cauchy distribution has none at all, not even a mean.