Shrinkage Covariance Estimation (Ledoit–Wolf) / Shrinkage LDA
The sample covariance matrix S is an unbiased but high-variance estimator; in high dimensions its extreme eigenvalues are biased (the largest inflated, the smallest shrunk toward zero), so inverting it amplifies noise. Shrinkage replaces S with a convex combination (1 − γ)·S + γ·ν·I, pulling eigenvalues toward the mean eigenvalue ν and conditioning the inverse. The shrinkage intensity γ in [0, 1] trades bias for variance.
Ledoit and Wolf derived an analytic, data-driven γ that minimizes the expected Frobenius error to the true covariance without any cross-validation. Substituting this regularized covariance into LDA yields shrinkage-LDA, the de facto standard linear decoder for ERP and oscillatory BCI: it is nearly parameter-free, robust when features outnumber trials, and typically matches or beats cross-validated ridge tuning at negligible cost.
For a 60-channel × 10-sample ERP feature vector (600 dimensions) trained on 200 trials, the sample covariance is singular and plain LDA is undefined; Ledoit–Wolf shrinkage yields an invertible estimate and a usable decoder.
Shrinkage makes a high-dimensional covariance invertible.
Shrinking toward the scaled identity is only one target; shrinking toward a diagonal of per-feature variances, or toward a covariance shared across classes, can be preferable when feature scales differ.