dall-e

DALL-E is OpenAI's line of text-to-image systems, and tracing its three generations is a compact tour of how the whole field evolved — from autoregressive transformers to diffusion. The original DALL-E (early 2021) was a landmark demonstration that a single model could generate coherent images from free-form captions and combine concepts in genuinely novel ways ('an armchair in the shape of an avocado'), proving text-to-image was a tractable, general capability rather than a narrow trick.

Technically the generations differ sharply. DALL-E 1 was autoregressive: a discrete VAE (dVAE) compressed images into a grid of code tokens, and a large transformer generated those image tokens one after another, conditioned on the text tokens — essentially 'language modeling over image patches.' DALL-E 2 (2022) switched to diffusion and the unCLIP architecture: a 'prior' model maps the text to a CLIP image embedding, then a diffusion 'decoder' generates an image from that embedding, with super-resolution diffusion stages upscaling the result. DALL-E 3 (2023) focused on prompt fidelity, and is notable for being trained on richly rewritten, highly descriptive synthetic captions and integrated with a large language model (GPT) that expands a user's short prompt into a detailed one before generation — dramatically improving how faithfully complex prompts are followed.

DALL-E matters both as a research lineage and as a reference point. It popularized the autoregressive-image-token approach (later echoed by other systems), then helped establish diffusion as the dominant paradigm, and DALL-E 3's caption-rewriting strategy reframed prompt-following as partly a data-and-language-model problem rather than purely a generator-architecture problem. Unlike Stable Diffusion, the DALL-E weights are closed and accessed via API, so it sits on the proprietary side of the open/closed divide that defines the modern landscape.