conditional variance

Variance measures how spread out a variable is around its mean. Conditional variance asks the same question, but after you have been told some information: once I know Y (or a sigma-algebra G), how much uncertainty about X is LEFT? It is the spread that survives conditioning — the residual unpredictability after the best estimate has been made.

By direct analogy with ordinary variance, Var(X given G) = E[ (X - E[X given G])^2 given G ], which (just like Var(X) = E[X^2] - (E[X])^2) can be rewritten as E[ X^2 given G ] - ( E[X given G] )^2. Like the conditional expectation it is a random variable: it can take different values depending on which slice of information you land in — knowing Y might shrink the spread of X a lot for some values of Y and little for others. It is always nonnegative (that is conditional Jensen with phi(x) = x^2), and it is zero exactly when X is already determined by G, since then there is nothing left to vary.

Conditional variance is the heart of the variance-decomposition identity, the law of total variance: Var(X) = E[ Var(X given Y) ] + Var( E[X given Y] ). In words, the total uncertainty in X splits cleanly into the average leftover uncertainty after knowing Y (the 'unexplained', within-group part) plus the variability of the conditional mean itself (the 'explained', between-group part). This is the probabilistic skeleton of the analysis-of-variance decomposition in statistics and of the bias-variance trade-off in machine learning. The caution: Var(X given Y) is a function of Y (a random variable), not a single number — average it with E[ · ] before comparing it to a plain variance.