Expectation, Variance & Moments

the moment generating function

Computing moments one at a time — E[X], E[X^2], E[X^3], ... — is tedious, and each is its own integral. What if a single function could store all the moments at once and hand any one of them over with a quick derivative? That is exactly what the moment generating function does: it is a moment factory.

It is defined as M_X(t) = E[e^(tX)], a function of a helper variable t. The trick is the power-series expansion e^(tX) = 1 + tX + (tX)^2/2! + (tX)^3/3! + ... ; taking expectations term by term, M_X(t) = 1 + t E[X] + t^2 E[X^2]/2! + ... So the moments are sitting in the coefficients. To extract the k-th moment, differentiate k times and set t = 0: E[X^k] = M_X^(k)(0). The first derivative at 0 gives the mean, the second gives E[X^2], and so on — one tidy object, all the moments.

The mgf also has a magical multiplication rule: for independent X and Y, the mgf of the sum is the product of the mgfs, M_{X+Y}(t) = M_X(t) M_Y(t). That turns the hard problem of adding random variables into easy multiplication, which is how one shows the sum of independent normals is normal. And when an mgf exists in a neighbourhood of 0, it uniquely determines the distribution — matching mgfs means matching distributions. The big caveat: the mgf need NOT exist, because E[e^(tX)] can diverge for heavy-tailed variables (the lognormal and Cauchy have no usable mgf). That is precisely why probabilists often switch to the characteristic function E[e^(itX)], which always exists.

For X ~ Exponential with rate lambda, M_X(t) = lambda/(lambda - t) for t < lambda. Differentiate once and set t = 0: M_X'(t) = lambda/(lambda - t)^2, so E[X] = lambda/lambda^2 = 1/lambda — the mean falls right out, no separate integral needed.

M_X(t) = E[e^(tX)]: differentiate k times at t=0 to read off the k-th moment.

An mgf may fail to exist (the integral diverges) for heavy-tailed distributions like the lognormal or Cauchy. The characteristic function E[e^(itX)] always exists, which is why it is preferred for general theory.

Also called
MGFmgfM_X(t)動差生成函數