Spike tokenization

The design choice of how to turn continuous or point-process neural signals into the discrete or continuous 'tokens' a transformer consumes. Three broad strategies dominate. Binning then patching converts spikes into a time-binned rate matrix and groups bins into patches (the Neural Data Transformer approach) — cheap, but sub-bin timing is lost. Per-spike tokenization represents each individual spike as a token carrying a learned unit-identity embedding plus a continuous timestamp encoded with rotary or Fourier features (POYO) — this preserves precise timing and naturally accommodates a variable, unordered set of neurons. Vector quantization learns a codebook and maps signal windows to discrete codes (LaBraM's neural tokenizer for EEG), giving a finite vocabulary amenable to masked-token prediction.

The tradeoffs are concrete. Per-spike tokens keep timing but produce long sequences that scale with firing rate; binning is compact but coarse; discrete VQ codes enable clean language-model-style objectives but discard fine amplitude and timing. Tokenization is also where the heterogeneity problem is solved or deferred: a per-unit identity embedding is exactly what lets one model span arrays with different neurons.