Scaling, Multimodality, and In-Context Adaptation

Do scaling laws hold for brains?

In language and vision, loss falls as a smooth power law in parameters and data, which is what makes 'just scale it' a rational strategy. For neural data the question is genuinely open. Studies of scaling laws for neural data — pooling more sessions into a large-scale corpus (POYO, LaBraM, BrainLM) — show that more data and parameters generally help, but the curves are noisy, the regimes are tiny, and no stable exponents have been pinned down.

L(N,D)\ \approx\ L_{\infty}+\Big(\tfrac{N_{c}}{N}\Big)^{\alpha_{N}}+\Big(\tfrac{D_{c}}{D}\Big)^{\alpha_{D}}

A Chinchilla-style scaling law: expected loss versus parameters N and data D. Written here only as a template — for neural data the exponents alpha_N, alpha_D and the irreducible term are NOT reliably established.

Multimodal models

A powerful frontier couples neural activity with other modalities in one model — a multimodal neural model over neural + behaviour + language. The most striking example is semantic decoding: aligning fMRI or intracranial activity to the embedding space of a large language model so that continuous meaning can be reconstructed (Tang and colleagues). This is brain-model representational alignment with a language model in the loop.

In-context and few-shot adaptation

The most elegant use of a pretrained backbone is in-context adaptation: hand the model a short labelled 'support set' from the new session and let it condition on it — no gradient step, no retraining. Close relatives are plug-and-play (zero-shot) decoding and meta-learning for rapid calibration, all measured on few-shot cross-session benchmarks.

p_{\theta}\big(y_{\star}\mid x_{\star},\ \{(x_{i},y_{i})\}_{i=1}^{n}\big)\qquad\text{(no update to }\theta\text{)}

In-context decoding: the frozen model predicts a new query's label by conditioning on a handful of support examples from the session, turning calibration into a forward pass instead of a training run.

Encoding models and digital twins

Not every foundation model decodes; some encode. An encoding foundation model predicts how neurons respond to arbitrary stimuli, and once trained becomes a queryable neural digital twin you can probe in silico — for example, foundation models of the mouse visual cortex that generalize to stimulus types they were never trained on. Encoding twins are also the natural bridge to the biomimetic-write problem: to write a percept you must first model how the brain would have encoded it.