Generating & Characteristic Functions

the characteristic function

The mgf is a wonderful tool but a fragile one: for heavy-tailed distributions it simply fails to exist, because E[e^(tX)] can blow up to infinity. Probabilists wanted a version of the same idea that never breaks — a single function encoding the whole distribution that is guaranteed to exist for every random variable, with no fine print. The characteristic function is exactly that: the always-available, fully rigorous cousin of the mgf.

It is defined as phi_X(t) = E[e^(itX)], where i is the imaginary unit (i^2 = -1). The one-character change from the mgf — putting an i into the exponent — is what tames it. Because e^(itX) = cos(tX) + i sin(tX) by Euler's formula, the quantity e^(itX) always has absolute value exactly 1; it is a point spinning on the unit circle, never growing. So the expectation E[e^(itX)] is an average of bounded things and is guaranteed finite, no matter how heavy the tails. In the language of analysis, phi_X(t) is the Fourier transform of the distribution, and that connection imports the whole powerful toolkit of Fourier analysis into probability.

Everything the mgf could do, the characteristic function does — and unconditionally. It multiplies for sums of independent variables, phi_{X+Y}(t) = phi_X(t) phi_Y(t); it has a uniqueness theorem (equal characteristic functions mean equal distributions); it has an inversion formula recovering the distribution; and its derivatives at 0 still generate moments, as far as those moments exist. Its one extra superpower is continuity: convergence of characteristic functions corresponds to convergence in distribution (Levy's theorem), which is precisely the lever that lifts the central limit theorem into a clean proof. The mild price is that it is complex-valued, so you carry an imaginary part along; but that small bookkeeping cost buys guaranteed existence.

For X ~ Normal(0, 1), the characteristic function is phi_X(t) = e^(-t^2/2) — real, smooth, and bounded. Even for the Cauchy distribution, which has no mean and no mgf at all, the characteristic function exists and is the clean phi_X(t) = e^(-|t|), showing the tool works exactly where the mgf gives up.

phi_X(t) = E[e^(itX)]: the Fourier transform of the distribution, bounded by 1 and always defined.

Unlike the mgf, the characteristic function always exists for every random variable, because |e^(itX)| = 1 keeps the expectation finite. The price is that it is complex-valued.

Also called
char. functionFourier transform of the distributionphi_X(t)特性函數