classifier-free guidance

Classifier-free guidance (CFG) is the single most important trick for making text-to-image models actually obey the prompt. A raw conditional diffusion model technically uses the text, but often only loosely — generations drift, ignore details, or look generic. CFG amplifies the influence of the prompt by, at every denoising step, comparing what the model wants to draw with the prompt against what it would draw with no prompt at all, and then exaggerating the difference. It is like asking 'how does mentioning a red bicycle change your guess?' and then leaning harder in that direction.

Mechanically, you train one network to handle both cases by randomly dropping the conditioning (replacing the prompt with a null/empty embedding) on, say, 10% of training examples. At sampling time you run the network twice per step — once with the prompt (conditional prediction epsilon_c) and once with the null prompt (unconditional prediction epsilon_u) — and combine them: epsilon_guided = epsilon_u + w * (epsilon_c - epsilon_u). The vector (epsilon_c - epsilon_u) points in the direction that conditioning pushes the image; the guidance weight w controls how far you push. With w = 0 you ignore the prompt; w = 1 is ordinary conditional sampling; w > 1 over-emphasizes the prompt.

The name 'classifier-free' is historical: it replaced an earlier method, classifier guidance, which steered samples using the gradient of a separate image classifier. That required training an extra noise-robust classifier and was fiddly. CFG achieves the same prompt-strengthening effect using only the generator itself, by treating the conditional/unconditional pair as a built-in implicit classifier. The cost is doubling the per-step compute (two forward passes), which several recent methods try to distill away, but the quality and controllability gains made CFG universal across Stable Diffusion, DALL-E-class systems, Imagen, and beyond.