denoising diffusion probabilistic models (DDPM)

A DDPM treats generation as undoing a fixed corruption process. The forward chain takes a clean datum and, over many steps, adds a little Gaussian noise each time, until after enough steps the data is indistinguishable from standard noise. Generation reverses this: the model starts from noise and, step by step, predicts and removes a small amount of noise, gradually sculpting structure out of randomness. Each reverse step is a tiny denoising problem, which is far easier to learn than producing a whole image in one shot.

The forward process is a Markov chain of Gaussians with a fixed variance schedule, and a convenient property is that the noisy sample at any step has a closed-form Gaussian given the clean datum, so training needs no simulation of the chain. Ho and colleagues showed that the variational bound on the data likelihood simplifies, under a reparameterization, to a plain mean-squared error between the true injected noise and a network's prediction of it. Sampling then applies the learned reverse transitions, which are Gaussian with means determined by the noise prediction.

DDPM is mathematically equivalent to discrete-time score-based diffusion: predicting the injected noise is, up to scaling, predicting the score of the noised marginal. This equivalence is what lets the same trained model be sampled by the slower stochastic chain, by deterministic implicit samplers, or by ODE solvers.