Sequential variational autoencoder (seq-VAE)

A sequential VAE is a VAE whose latent code is itself a trajectory governed by a dynamical prior. A generative decoder maps a low-dimensional latent state sequence to the observed neural time series (often via a Poisson emission for spikes), a dynamics prior — an RNN, a linear system, a neural ODE — constrains how the latent evolves, and an amortized inference (recognition) network approximates the posterior over the whole trajectory from the data. Training maximizes an evidence lower bound (ELBO) on the sequence likelihood.

In systems neuroscience LFADS is the canonical instance, using a generator RNN and an encoder that also infers a per-trial initial condition and time-varying inputs; many successors relax or swap these pieces. The framework is powerful because it denoises single-trial population activity into smooth latent trajectories and can infer unobserved inputs, but it inherits VAE pathologies — posterior collapse, sensitivity to the KL weighting and to the capacity of the inference network, and the ever-present risk that an over-flexible decoder or an inferred-input channel explains the data by memorizing rather than by discovering dynamics.