query, key, and value

Query, key, and value are the three roles every item plays inside an attention mechanism, and the cleanest analogy is a library search. The query is what you are looking for — the search terms you type. The key is the label on each book's spine that the search matches against. The value is the actual content inside the book you pull off the shelf. Attention compares your query to every key to decide which books are relevant, then hands you a blend of their values weighted by how well each matched.

In a Transformer, each token's vector is multiplied by three separate learned weight matrices to produce its own query, key, and value vectors. A given token uses its query to score against every token's key (including its own); those scores become attention weights; and the token then receives the weighted sum of all the values. The split is deliberate: keeping "what I'm searching for" (query) separate from "what I advertise" (key) and "what I deliver" (value) gives the model the freedom to match on one basis but pass along something different.

This trio is the vocabulary that makes the rest of attention precise. Self-attention is just the case where queries, keys, and values all come from the same sequence; cross-attention is when queries come from one sequence and keys and values from another. Understanding QKV is the difference between attention feeling like magic and seeing it for what it is: a learned, content-based lookup followed by a weighted average. The matrices are learned, so what counts as a good "match" is itself shaped by training, not fixed in advance.