the multinomial distribution
/ mul-tee-NOH-mee-al /
The multinomial distribution is the binomial set free from having only two outcomes. Instead of each trial being success or failure, each trial now lands in one of m categories. Roll a die n times and count how many 1s, 2s, ..., 6s appeared. Survey n voters and count how many favour each of three candidates. Classify n customers as new, returning, or churned. The result is not a single number but a vector of counts (X1, X2, ..., Xm) that must sum to the fixed total n.
Each category c has its own probability p_c, with all the p_c summing to 1, and the trials are independent and identical. The joint pmf generalizes the binomial in the obvious way: P(X1 = k1, ..., Xm = km) = (n! / (k1! k2! ... km!)) p1^k1 p2^k2 ... pm^km, valid when the k's are non-negative integers summing to n. The product of powers gives the chance of one specific sequence of outcomes; the leading multinomial coefficient n!/(k1! ... km!) counts how many distinct orderings produce those same category totals. Each individual count is itself binomial — X_c by itself is Binomial(n, p_c) with mean n p_c — but the counts are not independent: they are tied together by their fixed sum, so the categories are negatively correlated. If one count comes out high, the others must compensate, and indeed Cov(X_i, X_j) = -n p_i p_j for i not equal to j.
The multinomial is the workhorse behind categorical data: chi-squared goodness-of-fit tests, contingency tables, the bag-of-words model in text analysis, and the genetics of multiple alleles all rest on it. When m = 2 it collapses back to the binomial, confirming that the binomial is just the two-category special case.
Roll a fair die 12 times. The chance of getting exactly two of each face (k1 = ... = k6 = 2) is (12! / (2!)^6) (1/6)^12 which is about 0.0034. Each face's count alone is Binomial(12, 1/6) with mean 2.
A vector of counts summing to n; each margin is binomial, but the categories are negatively correlated.
The category counts are NOT independent — fixing the total n at the start forces a high count in one category to be offset elsewhere, giving negative covariance. Only each count's own margin is binomial.