Foundation Models & Representation Learning for Neural Data

Channel-agnostic architecture

A model that accepts a variable number and identity of input channels or neurons without a fixed, ordered input layer, so a single model can span arrays with different layouts and yields. The trick is to treat the population as an unordered set: each unit becomes a token carrying a learned identity embedding, and attention — often cross-attention into a fixed-size latent bottleneck, in the PerceiverIO style used by POYO — is invariant to the number and order of inputs. This is the architectural enabler behind cross-session and cross-subject foundation models.

Contrast this with classical decoders (a Kalman filter, or an MLP on a fixed neuron vector), which require the same neurons in the same order every day and break the moment the channel set changes. Making the input a set rather than a vector is precisely what lets pretraining pool many incompatible recordings and what lets a new session be onboarded by adding embeddings rather than rebuilding the model.

Also called
variable-input modelpermutation-invariant neural modelunit-set model