image generation from LLMs

So far most multimodal LLMs only read images; this is about making them. You type a watercolour fox reading a newspaper in the rain and get a picture. The language model's role is to understand the request — including a long, nuanced, multi-part description — and steer a separate image generator, usually a diffusion model, toward exactly that.

Two patterns dominate. In the coupled approach the LLM stays a text model and hands a rich prompt, or learned conditioning vectors, to a diffusion generator that paints the pixels; the LLM's strength is turning vague or conversational requests into precise instructions and editing across turns. In the unified approach the same transformer predicts image tokens directly, then a decoder renders them — letting one model interleave words and pictures in a single stream, useful for illustrated answers.

These systems render plausibility, not truth: counts, text within the image, anatomy, and faithful edits such as change only the hat are classic failure points, and they raise real questions about copyright, deepfakes, and consent that text generation does not.