query-key-value

Query, key and value are the three roles every token plays inside attention, and they make the whole mechanism feel like a soft, differentiable dictionary lookup. The analogy: a query is the search term you type, a key is the label on each entry that your search is matched against, and a value is the actual content stored under that entry. You compare your query to all the keys to decide how much each entry matches, then retrieve a blend of the values weighted by those matches. Attention is exactly this lookup, but soft — instead of returning one entry, it returns a weighted mixture of all of them.

Where do they come from? From the same input. Each token's embedding vector is multiplied by three separate learned weight matrices — W_Q, W_K and W_V — to produce its query, key and value vectors. These matrices are the parameters attention learns: they decide what aspect of a token it advertises as a key, what it asks for as a query, and what payload it offers as a value. Separating the three is the key idea — a token can search for one property while offering a different one, so 'what makes me findable' and 'what I deliver when found' are decoupled.

Once projected, the interaction is mechanical: scores come from query-dot-key, weights from a softmax over those scores, and the output from those weights applied to the values. In self-attention all three matrices read the same sequence; in cross-attention the query comes from one sequence and the key and value from another, which is how a decoder pulls information from an encoder or a text prompt conditions image features. Same three roles, different sources.

A practical wrinkle from large-scale deployment: keys and values dominate memory at inference because they must be cached for every past token (the KV cache), and they can be shared across heads to save memory — multi-query attention uses one shared key/value for all heads, grouped-query attention uses a few. These variants change only how many distinct K/V projections exist, not the fundamental query-key-value idea.