the Cauchy-Schwarz inequality for random variables
/ KOH-shee SHVARTS /
There is a classic fact about ordinary vectors: the dot product of two vectors is never bigger (in size) than the product of their lengths, |u . v| <= |u| |v|. It encodes that two arrows can align only so much. Random variables behave like vectors too — you can 'add' them, 'scale' them, and define a kind of length and angle — so the same inequality holds, and it is one of the most-used estimates in all of probability.
The statement: for random variables X and Y, (E[XY])^2 <= E[X^2] E[Y^2]. Think of E[XY] as a dot product, and E[X^2] as the squared length of X. The inequality says correlation cannot exceed the product of the magnitudes. A clean way to see it: for every real t, E[(X - tY)^2] >= 0 because a square is never negative. Expand to E[X^2] - 2t E[XY] + t^2 E[Y^2] >= 0, a quadratic in t that stays non-negative, so its discriminant cannot be positive: (2 E[XY])^2 - 4 E[Y^2] E[X^2] <= 0, which rearranges exactly to the claim.
Two payoffs follow immediately. First, applied to centred variables it proves the correlation coefficient always lies between -1 and 1, since |Cov(X,Y)| <= sigma_X sigma_Y. Second, it guarantees E[XY] is finite whenever X and Y each have finite second moments — a workhorse fact when juggling covariances and inner products. Equality holds only when one variable is a constant multiple of the other (perfect alignment), the exact case of correlation +1 or -1.
Let X have E[X^2] = 4 and Y have E[Y^2] = 9. Then no matter how X and Y depend on each other, |E[XY]| <= sqrt(4 . 9) = 6. Their average product is pinned inside [-6, 6]; it can only reach the ends if Y is exactly proportional to X.
Treat random variables as vectors: their 'dot product' E[XY] is capped by the product of their 'lengths'.
Cauchy-Schwarz bounds the LINEAR alignment E[XY]; reaching the bound means exact proportionality. It says nothing about general dependence — uncorrelated variables can still be strongly, non-linearly related.