Hyperparameter Optimization / Model Selection

The search over non-learned configuration — regularization strength, number of CSP components or selected channels, frequency bands, kernel width, network depth — that governs a pipeline's capacity. Strategies range from grid search (exhaustive over a discretized space, simple but scaling exponentially with the number of hyperparameters), through random search (often more efficient when only a few hyperparameters matter), to Bayesian optimization (a surrogate model proposes promising configurations, valuable when each evaluation is costly).

The inviolable rule is that selection must use only training data via an inner validation loop; reporting the best configuration's own validation score as performance is optimistic bias, and the fix is nested cross-validation. With small BCI calibration sets the selection itself is noisy, so a coarse search over a well-chosen prior (such as shrinkage LDA with analytic γ) often beats an exhaustive search that overfits the validation folds.