denoising diffusion probabilistic model
The denoising diffusion probabilistic model (DDPM) is the 2020 formulation by Ho, Jain, and Abbeel that made diffusion practical and ignited the field. Its central discovery is almost surprisingly humble: you can throw away most of the intimidating probability theory and train the model with a plain mean-squared-error loss. Concretely, take a clean image, corrupt it with a known amount of Gaussian noise to a random timestep, and ask a network to guess what noise you added. That single regression objective, averaged over all images and timesteps, is enough to learn a powerful image generator.
Formally, DDPM derives its training objective as a variational bound on the data log-likelihood (the same machinery behind variational autoencoders), but then simplifies it to the loss L = E[ || epsilon - epsilon_theta(x_t, t) ||^2 ]. Here epsilon is the true Gaussian noise drawn during corruption, x_t is the noised image at timestep t, and epsilon_theta is the network's prediction. The weighting that the full variational bound would assign to each timestep is dropped (set to one), which the authors found empirically improves sample quality — this is the famous 'simple' objective. At sampling time, the trained epsilon-predictor is plugged into an ancestral sampler that walks the reverse chain step by step, injecting a small amount of fresh noise at each step to keep the process stochastic.
DDPM matters because it is the canonical reference point against which everything else is described: DDIM is 'DDPM but deterministic and faster,' classifier-free guidance is 'DDPM conditioning made stronger,' latent diffusion is 'DDPM run in a compressed space.' Understanding DDPM's forward corruption, simple loss, and ancestral reverse sampler gives you the vocabulary for the entire diffusion literature. Its limitation — needing hundreds or a thousand sequential steps for best quality — is precisely the problem that the subsequent decade of research has chipped away at.
One DDPM training step: sample image x_0 from the dataset, sample t uniformly from {1,...,1000}, sample noise epsilon ~ N(0, I), form x_t = sqrt(alpha_bar_t) x_0 + sqrt(1 - alpha_bar_t) epsilon, run epsilon_theta(x_t, t), and take a gradient step on || epsilon - epsilon_theta ||^2. No discriminator, no adversary — just regression.
A subtle but important point: epsilon-prediction, x_0-prediction (directly predicting the clean image), and v-prediction (a velocity that interpolates between the two) are all reparameterizations of the same model and can be converted into one another. They differ mainly in numerical conditioning across timesteps — v-prediction, for instance, behaves better at high noise levels and is preferred for some distillation and high-resolution setups.