amortized variational inference

Classical variational inference fits a separate set of variational parameters for every datapoint, solving a little optimization problem per example. That is wasteful when you have millions of points. Amortized inference instead trains one shared neural network, an encoder, that looks at any input and predicts its posterior parameters in a single forward pass. You pay an up-front training cost once, and inference for a new point is then just a function evaluation.

The inference network, the recognition model, is shared across all datapoints, and its weights are optimized jointly with the generative model by maximizing the ELBO, using the reparameterization trick for low-variance gradients. This is exactly the encoder of a variational autoencoder; it amortizes the cost of inference over the whole dataset and, as a bonus, generalizes to unseen inputs at test time without any per-example optimization.

The win is speed and scalability, constant-time inference that is the backbone of variational autoencoders and many deep latent-variable models. The cost is the amortization gap: a single network cannot output the exact optimal variational parameters for every input, so the approximate posterior is systematically looser than per-datapoint variational inference would achieve. Semi-amortized methods refine the encoder's output with a few gradient steps to close part of that gap.