Masked neural modeling

Masked modeling pretrains an encoder by hiding part of the input — random time bins, whole channels, or channel-time patches — and training the network to reconstruct the missing values from the visible context, the neural-data analogue of masked language modeling and masked autoencoders. It underlies NDT-style intracortical models (mask spike bins, predict firing rates) and several EEG or iEEG foundation models (mask patches, predict the signal or its spectrogram).

Because it reconstructs the signal directly rather than making a contrastive comparison, masked modeling sidesteps the negative-sampling and positive-design problems of contrastive learning, but it can waste capacity reconstructing high-variance, behaviorally irrelevant components (line noise, slow drift, non-neural artifact) unless the target space or loss is chosen to emphasize the informative bands.