normalizing flows

A normalizing flow builds a complicated distribution by warping a simple one through an invertible function. Start from easy-to-sample noise such as a standard Gaussian, push it through a learnable bijection, and the output is your data distribution; run the bijection backwards and any datapoint maps cleanly back to its noise. Because the map is invertible and you can track exactly how it stretches and squeezes volume, you get something diffusion and GANs do not: an exact, computable likelihood for every point.

The machinery is the change-of-variables formula, which says the log-density of a transformed variable is the base log-density plus the log absolute determinant of the transformation's Jacobian. The design challenge is building expressive bijections whose Jacobian determinant is cheap to evaluate. Coupling layers update half the coordinates as a function of the other half, giving a triangular Jacobian whose determinant is a product of diagonal entries; autoregressive flows do likewise one coordinate at a time. Many such layers compose into a flexible flow trained by exact maximum likelihood.

Flows shine where calibrated densities matter, such as variational inference, simulation-based inference, and anomaly detection, but the invertibility-plus-tractable-Jacobian constraint caps their per-layer expressiveness, which is why they were partly eclipsed by diffusion for raw image fidelity.