Neural foundation model

A large model pretrained, usually self-supervised, on big and heterogeneous collections of neural recordings, producing general-purpose representations that transfer to many downstream subjects, brain areas, and tasks with little labeled data. The name is borrowed from the language and vision literature (Bommasani et al.'s 'foundation model'), and the aspiration is the same: pretrain once at scale, then adapt cheaply. Concrete instances include spiking-population models such as the Neural Data Transformer family and POYO/POYO+, intracranial and EEG models such as BrainBERT, BENDR and LaBraM, and embedding models such as CEBRA.

It is important to keep the aspiration separate from the evidence. Today's neural foundation models are orders of magnitude smaller in data and parameters than large language models, and demonstrated generality is mostly within one modality and one species, with light fine-tuning rather than true broad zero-shot transfer. Calling a model a foundation model is a claim about breadth of reuse that should be checked on held-out subjects and sessions, not just in-distribution.