Speech & Language Neuroprostheses

Language model in the loop

Nearly every high-performance speech or handwriting BCI pairs a noisy neural sequence model with a language model that encodes prior knowledge of which word sequences are plausible. The neural model emits per-time-step probabilities over phonemes or characters; a beam-search decoder fuses these with an n-gram or neural language model to output the most likely sentence, correcting many neural errors.

This is why raw per-symbol neural accuracy and final word error rate can differ by an order of magnitude, and why enabling or upgrading the language model can improve reported performance without any change to the neural signal.

The language model is a double-edged prior: it sharply reduces error on in-distribution language but can produce fluent, confident, wrong text (hallucination) and biases output toward its training corpus; it also blurs claims about what information the neural signal alone carries.

Also called
LM decodingbeam-search decoding語言模型解碼