Generating & Characteristic Functions

how the mgf generates moments

The moment generating function M_X(t) = E[e^(tX)] is named for one job above all others: it generates the moments. Computing E[X], E[X^2], E[X^3], one at a time means doing a fresh integral or sum every time. The mgf promises something better — store all the moments in one function once, then fish out any single moment with nothing harder than a derivative.

Here is the mechanism, and it is just calculus. Expand the exponential as a power series: e^(tX) = 1 + tX + (tX)^2/2! + (tX)^3/3! + ... Take the expectation of each term (legal because expectation is linear) and you get M_X(t) = 1 + t E[X] + t^2 E[X^2]/2! + t^3 E[X^3]/3! + ... The k-th moment is sitting in the coefficient of t^k/k!. To pull it out, differentiate M_X(t) exactly k times and set t = 0; every term except the one you want either vanishes or differentiates away, leaving E[X^k] = M_X^(k)(0). One derivative at zero gives the mean, two give E[X^2], and Var(X) = M_X''(0) - (M_X'(0))^2.

This is why the mgf is the moment workhorse: instead of separate integrals you do one derivative each, and often you do not even need to compute the integral — you just look up or derive the mgf for a known distribution and differentiate. The honest condition is that the mgf must exist (be finite) in some open interval around t = 0; only then is it smooth enough that these derivatives exist and the expansion is valid. For heavy-tailed variables where E[e^(tX)] blows up, this whole machinery is unavailable, and you turn to the characteristic function instead.

For X ~ Normal(mu, sigma^2), M_X(t) = e^(mu t + sigma^2 t^2 / 2). Then M_X'(t) = (mu + sigma^2 t) e^(mu t + sigma^2 t^2/2), so M_X'(0) = mu = E[X]. A second derivative at 0 gives M_X''(0) = sigma^2 + mu^2 = E[X^2], hence Var(X) = (sigma^2 + mu^2) - mu^2 = sigma^2 — exactly the parameters, with no integral done by hand.

Differentiate M_X(t) k times and set t = 0 to read off the k-th moment E[X^k].

Reading moments off the mgf only works if the mgf is finite in an open interval around 0. If E[e^(tX)] diverges for every t != 0 (heavy tails), there are no derivatives at 0 to take.

Also called
differentiating the mgfmoments from the mgfM_X^(k)(0)由動差母函數取動差