expectation-maximization (EM)
/ ek-spek-TAY-shun MAK-sih-mih-ZAY-shun /
Expectation-maximization is a clever bootstrap for the chicken-and-egg problems that show up whenever some of your data is missing or hidden. Imagine sorting a mixed pile of coins into two stacks by weight, but you don't know the typical weight of each type, and you don't know which stack each coin belongs to. If you knew the stacks, you could compute the average weights; if you knew the weights, you could assign the coins. EM breaks the deadlock by guessing one, using it to estimate the other, and going back and forth until both settle.
The two steps name the method. In the Expectation (E) step, you take your current guess of the model and use it to fill in the hidden information softly — not 'this coin is type A' but 'this coin is 70% likely type A, 30% type B.' In the Maximization (M) step, you take those soft assignments as if they were real and re-estimate the model to fit them best. Repeat. A guarantee makes this trustworthy: each full round can only improve (or hold) the fit to the data — the process never makes things worse.
Why it matters: EM is the standard engine behind fitting mixture models, training hidden Markov models, and handling missing data across statistics and machine learning. Its honest limits are two. First, 'never gets worse' is not 'finds the best': EM climbs to a nearby peak, which may be a local optimum, so the starting guess matters and people often run it from several starts. Second, it gives you a single best-fit setting, not a full sense of uncertainty — it is a tool for estimation, not for the wider Bayesian goal of mapping out everything you don't know.
Heights in a crowd of mixed adults, where you suspect two groups but didn't record sex. Start by guessing two average heights. E-step: for each person, compute how likely they belong to the 'shorter' versus 'taller' group given current guesses. M-step: recompute each group's average using everyone, weighted by those probabilities. After a few rounds the two averages lock onto sensible values and the soft memberships stabilize.
EM alternates between softly guessing the hidden labels (E) and refitting the model to those guesses (M) until it converges.
EM is guaranteed not to make the fit worse, but that is not a guarantee of finding the global best — it can get stuck on a local optimum that depends on where you started. Running it from several random initializations and keeping the best result is standard practice.