cross-attention conditioning

Cross-attention conditioning is the mechanism by which words actually reach into the image and shape it. Inside the denoising U-Net, after the model has built up features describing the image-in-progress, cross-attention lets every spatial location of that image 'look up' the text embeddings and ask 'which words are relevant to me, and how should they change what I am?' The region that will become a dog attends strongly to the token 'dog'; the region that will become sky attends to 'blue' and 'sky.' This per-location, content-dependent routing of textual meaning into spatial features is what makes a prompt steer composition rather than just set a global mood.

Attention works with three roles: queries, keys, and values. In cross-attention the queries come from the image features (each spatial position produces a query vector), while the keys and values come from the text embeddings — this is what makes it 'cross' (one source attends to a different source), as opposed to self-attention where all three come from the image. For each image position, the model computes similarity (dot product) between its query and every text key, normalizes these into attention weights with a softmax, and returns the weighted sum of the text values. That returned vector is added back into the image features, injecting the relevant textual information exactly where it is needed.

Cross-attention layers are interleaved throughout the U-Net at multiple resolutions, so text guidance acts on both coarse layout and fine detail. This design is also why text-to-image models are so editable: the attention maps reveal which image regions each word controls, enabling techniques like prompt-to-prompt editing (change a word, swap only the regions that word governs) and attention-based localization. It is the single most important architectural bridge between the language and vision halves of a text-to-image system.