probability distribution
/ prob-uh-BIL-uh-tee dis-truh-BYOO-shun /
A probability distribution is the full accounting of how likely every possible outcome is. If a random variable is the question 'what number will this uncertain thing be?', the distribution is the complete answer: it lays out each possible value alongside its share of the probability. Think of it as how a fixed budget of certainty — exactly 1, or 100% — gets spread across all the things that could happen.
For something with a few discrete outcomes, the distribution is just a list: a fair die spreads its budget evenly, 1/6 on each face. For continuous quantities like height, you instead draw a smooth curve where the area under any stretch tells you the probability of landing in that range, and the total area is exactly 1. The shape of the curve is the whole story — where it bulges, the values are common; where it is thin, they are rare. Famous shapes include the bell-shaped normal distribution and the all-or-nothing Bernoulli.
Machine learning is, at bottom, the business of learning distributions. A classifier outputs a distribution over labels ('80% cat, 15% dog, 5% fox'). A language model produces a distribution over which word comes next. Training nudges the model's distribution to match the patterns in real data. Recognizing that an output is a distribution, not a single hard answer, is what lets a model honestly express uncertainty — and what lets us measure, with tools like cross-entropy, how far its beliefs sit from the truth.
Flip two fair coins and count the heads. The outcomes spread out as: 0 heads with probability 1/4, 1 head with probability 1/2, 2 heads with probability 1/4. Those three numbers add up to exactly 1 and form the distribution — note the middle value is twice as likely, because there are two ways to get one head (HT and TH).
Two coin flips: probabilities 1/4, 1/2, 1/4 across 0, 1, 2 heads — a complete budget of certainty summing to 1.
For continuous distributions the curve's height is a density, not a probability — only the area under it gives a probability, which is why the chance of any single exact value is effectively zero. Every valid distribution's probabilities (or total area) must add up to exactly 1.