Deep state-space model (S4 / structured SSM)

A deep state-space model stacks classical linear state-space recurrences (x_t = A x_{t-1} + B u_t, y_t = C x_t) as trainable neural layers, so that a network can propagate information over very long sequences with the linear-time, parallelizable structure of a convolution rather than the sequential bottleneck of an RNN. The Structured State Space Sequence model (S4, Gu et al.) and its successors (S4D, S5, and the selective SSM Mamba) achieve this by parameterizing A with structure — notably the HiPPO initialization, which makes the hidden state an online orthogonal-polynomial memory of the input — and by computing the induced convolution kernel efficiently.

For neural interfaces the appeal is twofold: neural recordings are long, multichannel time series with dependencies spanning seconds, and structured SSMs give a principled long-memory backbone that is cheaper than attention at long context. Emerging work applies SSM backbones to EEG, ECoG and spiking data for decoding and self-supervised pretraining. It is worth separating two senses of 'state-space' that the field overloads: the deep-learning SSM is an architecture for sequence modeling, whereas the probabilistic latent state-space model (as in Kalman/EM inference) is a generative model of the neural process. S4-style layers are usually deterministic and do not by themselves yield calibrated uncertainty or an interpretable latent dynamical law.