Joint Distributions, Covariance & Correlation

the conditional distribution

Often you learn the value of one variable and want to update your view of another. If you are told today's rainfall, what does tomorrow's now look like? The conditional distribution of Y given X = x is exactly that updated distribution of Y once you fix the value of X.

It is the random-variable version of conditional probability. For discrete variables, the conditional pmf is p_{Y|X}(y | x) = P(X = x AND Y = y) / P(X = x) — restrict attention to the column where X = x, then rescale that column so its probabilities add to 1 again. The intuition: you have learned X = x, so you live only in the slice of the world where that is true, and you re-ask how Y is distributed within just that slice. There is one whole conditional distribution of Y for each possible value x.

Conditioning is how dependence becomes useful: it lets you predict, update, and reason in stages. If the conditional distribution of Y given X does not actually depend on x — if every slice looks the same — then X tells you nothing about Y, and that is precisely independence. The gap between the conditional and the plain marginal is the value of the information X carries.

Roll one die; let X be its value and Y the indicator that the value is even. Given X = 4, the conditional distribution of Y is degenerate: Y = 1 with probability 1. Given X = 3, Y = 0 with probability 1. The slice you condition on completely determines Y here.

Conditioning restricts to one slice of the joint, then rescales it to a valid distribution.

P(Y = y | X = x) needs P(X = x) > 0; for continuous X this is zero, so the conditional is defined through densities instead (see the conditional density).

Also called
conditional law條件分配