Joint Distributions, Covariance & Correlation

the law of total expectation

When a quantity is hard to average directly, it often helps to break the world into cases, average within each case, and then average those case-averages weighted by how likely each case is. The law of total expectation is the exact formula for that two-step averaging.

It says E[X] = E[ E[X given Y] ]. Read from the inside out: E[X given Y] is the average of X computed within each value of Y — a number that depends on which Y you got, hence itself a random variable. The outer E then averages that quantity over Y. In the discrete case this unpacks to E[X] = sum over y of E[X given Y = y] times P(Y = y): split into cases by Y, take the conditional mean in each case, and weight by the case probabilities. It is the expectation cousin of the law of total probability.

This is one of the most-used tools in applied probability. It powers calculations with mixtures and random sums, lets you compute a difficult expectation by conditioning on a convenient helper variable, and underlies regression (the conditional mean is the best predictor) and dynamic-programming-style recursions. The key safety condition is that the inner conditional expectations exist; the tower then assembles them honestly into the overall mean.

A hen lays N eggs, where N is Poisson with mean 10, and each egg hatches independently with probability 0.8. Let X be the number that hatch. Given N = n, E[X given N = n] = 0.8 n, so E[X given N] = 0.8 N. Then E[X] = E[0.8 N] = 0.8 times 10 = 8 hatchlings.

Condition on the helper N, take the conditional mean, then average over N.

E[X given Y] is a random variable (a function of Y), not a number; the outer expectation averages it over Y. Confusing it with the single number E[X given Y = y] is the usual slip.

Also called
tower propertylaw of iterated expectations全期望法則重期望公式