the Wishart sample covariance matrix
/ Wishart = WISH-art /
The sample covariance matrix is the random matrix of statistics. Whenever you estimate the covariance of p variables from n observations — in finance, genomics, signal processing, PCA — you form an empirical covariance matrix, and in the modern regime where p is comparable to n (not vanishingly small), that matrix is a genuinely random object whose eigenvalues are systematically distorted away from the truth. Random matrix theory's verdict on this distortion is the Marchenko-Pastur law; the matrix that produces it is the Wishart sample covariance matrix.
Take a p-by-n data matrix X whose columns are n independent samples of a p-dimensional vector, with iid entries of mean 0 and variance 1 (so the true population covariance is the identity). The sample covariance matrix is S = (1/n) X X^*, a p-by-p Hermitian matrix that estimates the identity. When the entries are Gaussian, S follows the Wishart distribution (the matrix generalisation of the chi-squared / gamma distribution); its joint eigenvalue density has a Vandermonde repulsion product |lambda_i - lambda_j|^beta times a product of lambda_i raised to a power times exp(-(n beta/2) lambda_i), the Laguerre weight — hence the alternative name Laguerre ensemble. The eigenvalues of S are the squared singular values of X/sqrt(n) and are all nonnegative. The crucial regime is the high-dimensional limit where both p and n tend to infinity with the aspect ratio y = p/n converging to a constant in (0, infinity); this is the regime in which S has a non-trivial limiting spectrum.
What is profound and counterintuitive is that even when the population covariance is exactly the identity — so the true eigenvalues are all 1 — the eigenvalues of the sample covariance matrix S do NOT cluster at 1. They spread out into the Marchenko-Pastur bulk on an interval whose width is governed by y = p/n. With p = n (y = 1), the sample eigenvalues spread all the way from 0 to 4 even though the truth is a flat 1. This is the central lesson for high-dimensional statistics: the sample covariance matrix is a badly biased estimator of the population covariance spectrum, and naive PCA on high-dimensional data is reading structure into pure noise. A caveat: the limiting law assumes the aspect ratio converges to a finite positive constant and the entries have finite variance; the classical statistical regime p fixed, n -> infinity (where S -> identity by the law of large numbers) is the degenerate case y -> 0, and only there does the estimator behave.
Simulate p = n = 1000: draw a 1000-by-1000 matrix X with iid N(0,1) entries and form S = (1/1000) X X^*. The true covariance is the identity, so every population eigenvalue is 1. Yet the histogram of the 1000 eigenvalues of S is not a spike at 1 — it is a smooth density spread from 0 to 4, the Marchenko-Pastur law at y = 1. The smallest sample eigenvalues are near 0 and the largest near 4, a 4-to-near-0 spread invented entirely by finite sampling.
Identity truth, smeared sample spectrum: the sample covariance matrix distorts eigenvalues when p ~ n.
S = (1/n) X X^* is an unbiased estimator of each entry of the population covariance, but it is a badly biased estimator of the covariance spectrum when p ~ n. The eigenvalues are spread, the smallest pushed toward 0 and the largest inflated — do not trust sample eigenvalues at face value in high dimensions.