Joint Distributions, Covariance & Correlation

the bilinearity of covariance

Covariance behaves like a kind of multiplication between random variables, and like multiplication it distributes over sums and pulls out constants — separately in each of its two slots. This property, called bilinearity, is what makes covariance and variance calculations mechanical rather than mysterious.

Spelled out: Cov(aX + b, Y) = a Cov(X, Y) — constants multiply through, and adding a constant b changes nothing because shifting a variable does not change how it co-varies. And it distributes: Cov(X + Z, Y) = Cov(X, Y) + Cov(Z, Y). The same holds in the second slot, so you can expand Cov of any linear combination by treating it term by term, just like multiplying out brackets: Cov(sum of a_i X_i, sum of b_j Y_j) = double sum of a_i b_j Cov(X_i, Y_j). Two useful anchors: Cov(X, X) = Var(X), and if X and Y are independent then Cov(X, Y) = 0.

Bilinearity is the lever behind almost every variance computation with several variables — most importantly the variance of a sum, and the standard error of an average. Linearity of expectation always lets you add means; bilinearity of covariance is the companion rule that lets you assemble variances, with covariance terms appearing exactly when the variables are not independent.

Var(X + Y) = Cov(X + Y, X + Y) = Cov(X, X) + Cov(X, Y) + Cov(Y, X) + Cov(Y, Y) = Var(X) + Var(Y) + 2 Cov(X, Y). The cross term 2 Cov(X, Y) drops out exactly when X and Y are uncorrelated.

Bilinearity lets you expand Var(X + Y) by multiplying out, exactly like brackets.

Independence gives Cov = 0, but Cov = 0 does not give independence — so you can build variances of sums from covariances, but a zero cross term means uncorrelated, not necessarily independent.

Also called
covariance is bilinear共變異數雙線性