queries, keys, and values

A helpful way to picture attention is a library lookup. Each token writes a query — what am I looking for right now? Every token also posts a key, a label advertising what it is about, and holds a value, the actual content it will hand over. To decide what to read, a token matches its query against all the keys and pulls back a blend of the values whose keys matched best.

These three are not the raw token vector; they are three separate learned projections of it. The same input is multiplied by three different weight matrices to produce a query, a key, and a value, each living in its own space. Separating them lets a token ask one kind of question while advertising a different kind of answer. The query-key match produces scores; the values are what actually get mixed and passed on.

The three names map cleanly onto the metaphor: queries seek, keys are sought, values are served. The keys and values for past tokens are exactly what an LLM caches at inference time, so it need not recompute them for every new word it generates.