diffusion model
Imagine taking a sharp photo and slowly stirring in static (random noise), frame by frame, until nothing is left but pure television snow. A diffusion model learns to play that movie backwards: starting from snow, it repeatedly wipes away a little noise until a coherent image emerges. The crucial trick is that adding noise is trivial and has no learnable parameters, whereas removing it is hard — so we only ever train a neural network to perform the hard direction, and we let it take many small, easy steps rather than one impossible leap.
Precisely, a diffusion model defines two processes over a sequence of timesteps t = 0, 1, ..., T. The forward process q gradually corrupts a clean image x_0 into noise x_T by adding small amounts of Gaussian (bell-curve) noise according to a fixed schedule. The reverse process p_theta, carried out by a neural network with parameters theta, learns to undo one step at a time: given a noisier image x_t it predicts the distribution of the slightly cleaner image x_{t-1}. Because each forward step perturbs the image only a little, each reverse step is approximately Gaussian and therefore learnable. In practice the network is trained to predict the noise that was added (called epsilon-prediction), which is mathematically interchangeable with predicting the clean image or the score (the gradient of the log-density).
Diffusion models displaced GANs (generative adversarial networks) as the dominant method for high-fidelity image synthesis around 2021-2022. They win because training is a simple, stable regression loss with no adversarial min-max game, they cover the data distribution well (much less mode collapse, where a generator produces only a few kinds of output), and they scale predictably with data and compute. Their chief drawback is slow generation, since sampling chains many sequential network evaluations together; this motivated fast deterministic samplers (DDIM) and latent-space variants (latent diffusion, Stable Diffusion). Essentially every modern text-to-image system is a diffusion model at heart.
To generate a 512x512 picture: draw x_T as pure Gaussian noise, then loop t = T down to 1, each time running the network to estimate the noise in x_t and subtracting a calibrated portion of it to obtain x_{t-1}. After, say, 50 steps you reach x_0, a clean synthesized image. The same network weights are reused at every step; only the timestep input changes.
A common misconception is that the network 'denoises in one shot.' It does not — a single forward pass gives only an estimate of the noise (or clean image), and quality comes from iterating that estimate over many steps. Asking a model trained this way to jump from pure noise to a final image in one step generally yields a blurry average; recovering fidelity is exactly what the multi-step reverse chain (or modern few-step distillation) is for.