score-based diffusion

Imagine taking a clean image and gradually stirring in noise until it becomes pure static; score-based diffusion learns to run that movie backwards. The score is the gradient of the log-density of the noisy data, and at every noise level it points toward where probability mass is denser, that is, toward more plausible data. If you knew that direction everywhere, you could start from random noise and repeatedly nudge a sample uphill in density until it lands on something that looks like a real datum.

Formally, a fixed forward stochastic process perturbs data into a tractable prior, usually a Gaussian. Song and colleagues unified the construction as a stochastic differential equation whose time reversal, by Anderson's 1982 theorem, depends only on the score of the marginal at each time. A single network is trained to approximate the score field across all noise scales, after which sampling integrates the reverse SDE or its probability-flow ODE from prior noise back to data. The crucial fact is that the score of the noised marginals is learnable by denoising score matching even though those marginals are themselves intractable.

This view unifies discrete-time denoising diffusion, noise-conditional score networks, and continuous-time formulations, and it underlies modern image, audio, and molecular generators. By decoupling the model, a score field, from the sampler, any SDE or ODE solver, it enables guidance, fast few-step samplers, and exact likelihood evaluation through the change-of-variables formula.