Algorithmic Mediation of Intention
Every BCI output passes through a statistical model that interprets neural activity, so what reaches the world is not raw intention but the decoder's inference about it. This mediation is unavoidable and often beneficial — priors, smoothing, and language models improve speed and accuracy — but it means the model's training data, assumptions, and errors shape the user's expressed will. Biases in the data can systematically distort outputs; a strong language prior can substitute a probable phrase for the one the user meant.
The concern intensifies as models grow more predictive and 'helpful.' A speech neuroprosthesis coupled to a large language model may complete, correct, or reword decoded fragments, improving fluency while raising the risk that the system speaks for the user rather than through them. Governance responses include keeping the user in the loop with review-and-confirm steps, exposing and bounding the model's contribution, and auditing decoders for bias.
In text or speech BCIs, an aggressive language-model prior can push an ambiguous decode toward the most statistically common completion — helpful when it is right, but a subtle case of the machine, not the user, choosing the words when it is wrong.
The stronger the prior, the blurrier the line between assistance and speaking for the user.