Joint Distributions, Covariance & Correlation

independence of random variables

Two random variables are independent when learning the value of one tells you nothing about the other. Knowing how a die in Tokyo landed should not change your beliefs about a coin in Lima — those outcomes carry no information about each other. Independence is the precise version of that idea.

Formally, X and Y are independent if for every choice of ranges, P(X is in A AND Y is in B) = P(X is in A) times P(Y is in B). Equivalently, the joint distribution factors into the product of the two marginals everywhere, and the conditional distribution of Y given X equals the plain marginal of Y for every x — the slices all look the same. Any one of these statements implies the others; they are different faces of the same property.

Independence is enormously convenient: it makes the variance of a sum just add up, lets you multiply rather than wrestle with dependence, and underlies the entire theory of i.i.d. samples (independent and identically distributed), the law of large numbers, and the central limit theorem. But beware two traps. Independent is NOT the same as mutually exclusive: two events of positive probability that cannot both happen are actually maximally DEPENDENT, not independent. And pairwise independence (every pair independent) does not imply mutual independence of the whole collection.

Flip a coin and roll a die separately. P(heads AND a 6) = (1/2)(1/6) = 1/12 — exactly the product, so the two are independent. But the events 'heads' and 'tails' on a single coin are mutually exclusive AND highly dependent: knowing one is heads tells you the other did not happen.

Independence means probabilities multiply; mutual exclusivity is the opposite of independence, not the same.

Independent is not mutually exclusive. Two events of positive probability cannot be both — disjoint events are dependent because one happening forbids the other.

Also called
independent random variables獨立隨機變數