Probabilistic & Bayesian ML

prior, posterior, and likelihood

/ PRY-or, pos-TEER-ee-or, and LY-kli-hood /

These three words are the cast of characters in every Bayesian update, and they map onto a simple story. The prior is what you believed before looking at today's data. The likelihood asks, for each possible explanation, how well does it account for the data I just saw? The posterior is your belief afterward — the prior reshaped by the evidence. In one line: posterior is proportional to prior times likelihood.

A weather example makes it concrete. Your prior might be that any given day in your town has a 30% chance of rain. The likelihood describes how a dark, heavy sky would arise under 'rain today' versus 'no rain today' — dark skies are far more likely when it's about to rain. After you glance outside and see that sky, your posterior chance of rain jumps well above 30%. The prior was your baseline; the sky was the evidence; the posterior is the updated forecast.

Two honest cautions keep these terms from being misused. First, the likelihood is not a probability of the explanation — it is the probability of the data given each explanation, read sideways. Second, the prior is a genuine assumption, not a neutral starting point: with lots of data the prior fades and the likelihood dominates, but with little data the prior can quietly steer the answer. Naming all three forces you to say out loud what you assumed, what you observed, and what you now believe.

You flip a new coin and want its bias toward heads. Prior: you assume it's probably fair, peaked near 0.5. Likelihood: you flip 10 times and get 7 heads — the data favor biases a bit above 0.5. Posterior: a blend, perhaps centered near 0.6, still wide because 10 flips is little data. Flip 1,000 times with 700 heads and the posterior tightens hard around 0.7, the prior all but forgotten.

More data shrinks the posterior and drowns out the prior; with little data, the prior still speaks loudly.

A frequent slip is treating the likelihood as a probability distribution over explanations — it is not. The likelihood is a function of the explanation but a probability of the data; it need not sum to one across explanations. Only the posterior is a proper probability over what you want to know.

Also called
prior distributionposterior distributionlikelihood function先验分布后验分布似然先驗分布後驗分布