Transformer for neural data

Transformers apply self-attention — a mechanism that, for each element of a sequence, computes a weighted combination of all other elements based on learned query-key similarity — to neural time series and neural populations. Attention removes the recurrent bottleneck: it processes a window in parallel and can directly relate distant time points or distant electrodes, which suits the long, multichannel context of ECoG speech decoding and multi-region recordings. Positional encodings restore the temporal (or spatial) ordering that attention is otherwise blind to.

For neural data the challenge is data hunger: attention has weak inductive bias and many parameters, so from-scratch transformers overfit the modest trial counts of most BCI datasets. Their success has therefore come mainly through self-supervised pretraining and foundation-model recipes, and through domain-specific tokenizations (per-channel patches, spike-count bins) that inject the structure the vanilla architecture lacks.