normalizing-flow posteriors

Mean-field variational inference assumes the posterior is a product of independent Gaussians: cheap, but unable to capture correlations or curved, multi-modal shapes. Normalizing-flow posteriors fix this by starting from a simple Gaussian and pushing it through a chain of invertible, differentiable transformations, bending it into a far richer distribution while still being able to evaluate its density exactly.

Each transformation's effect on density is tracked by the change-of-variables formula, so the log-density of the warped sample is the base log-density minus the sum of the log-determinants of the Jacobians. Flows such as planar, inverse-autoregressive (IAF), RealNVP, and neural spline flows are engineered so these Jacobian log-determinants are cheap. Rezende and Mohamed used them to make the variational q flexible inside the ELBO: you sample base noise, flow it, and the reparameterization gradient passes straight through.

This sharpens the variational approximation toward the true posterior, reducing the ELBO's looseness, and is central to expressive variational autoencoders (via inverse-autoregressive flows) and to flexible weight or latent posteriors. The cost is added parameters and compute per flow step, plus a depth-versus-expressiveness trade-off: too few layers and you are back near mean-field, too many and the optimization grows fragile.