Ecological validity

The degree to which a result obtained in the laboratory generalises to the conditions, tasks, and behaviour of real life. For BCI it is the question that decides whether a demonstration matters: a decoder that works when a seated user attends to a screen in a shielded room may collapse when the same user is walking, distracted, tired, in electrical noise, or simply trying to do something they actually want to do.

Ecological validity is often in tension with internal validity — the tight experimental control that lets you attribute an effect to a cause. Lab studies buy internal validity by removing exactly the variability that real deployment must survive. Taking ecological validity seriously means testing in context and against realistic distractors and baselines, and reporting the drop from lab to field rather than the lab number alone; the size of that drop is frequently the most informative statistic a paper can give.