Curse of Dimensionality

As feature dimension p grows, the volume of feature space grows exponentially, data become sparse, distances concentrate, and the number of parameters to estimate (a p×p covariance has about p^2/2 free entries) explodes relative to the number of trials n. When p is comparable to or exceeds n, the sample covariance is singular, empirical class means are unreliable, and unregularized classifiers fit noise. The sample size required for a fixed accuracy grows roughly with dimension.

BCI is chronically in the p ≳ n regime: hundreds of time-by-channel features against tens to a few hundred calibration trials. The field's responses are dimensionality reduction (CSP, xDAWN, PCA, band-power summaries), strong regularization or shrinkage, and priors that constrain the effective number of parameters.