Class Imbalance
When classes occur at unequal frequencies — common in ErrP detection (few errors), P300 (rare targets among frequent non-targets, often 1:5 or worse), or seizure and event detection — a classifier can maximize accuracy simply by predicting the majority class, so raw accuracy becomes meaningless. The signal of interest is precisely the rare class.
Remedies operate at three levels: metrics (use balanced accuracy, precision/recall, F1, ROC-AUC, or Cohen's kappa rather than accuracy), data (resampling — undersample the majority, oversample the minority, or synthesize with SMOTE), and algorithm (class-weighted loss, an adjusted decision threshold, or setting LDA's priors to the operational rather than the training class frequencies). The right decision threshold depends on the application's cost of a miss versus a false alarm, which should be stated explicitly.
A P300 speller sees about one target per six flashes. A detector labeling everything non-target is 83% accurate yet useless; the speller needs high target recall at a controlled false-positive rate, evaluated by area under the ROC curve or by character-selection accuracy after evidence accumulation.
Under imbalance, accuracy is misleading.
Resampling must occur inside the cross-validation training fold only; oversampling before the split copies minority instances into both train and test and is a classic leakage.