Queries, Keys, and Values: The Arithmetic of Attention

Three roles for every word

Here is the central trick. From each word's vector the model produces three new vectors by multiplying it with three learned weight matrices. These are the word's query, key, and value. A library analogy makes them concrete:

1. Query (Q) — what I am looking for right now. The word it sends out a query like *"I'm a pronoun hunting for the noun I refer to."*
2. Key (K) — what I advertise about myself, like the label on a library shelf. trophy advertises *"I'm a concrete noun."*
3. Value (V) — the actual content I'll hand over if you decide to attend to me — the book itself, not the label.

Scoring: who is relevant to whom

To find out how relevant word B is to word A, we compare A's query with B's key using a dot product — multiply the two vectors element by element and add up the results. A large dot product means the query and key point in similar directions: a strong match. A small or negative one means they're unrelated. The resulting number is an attention score.

Each word computes a score against every word in the sentence, itself included. For a 10-word sentence that's a 10×10 grid of scores. This whole operation — query dotted with keys, then used to mix values — has a name you'll see everywhere: scaled dot-product attention. The scaled part divides the scores by a constant so they don't blow up when the vectors are long, which keeps the next step stable.

From scores to weights with softmax

Raw scores can be any number. We need them turned into clean proportions that sum to 1 — *"spend 70% of your attention on trophy, 20% on suitcase, 10% on the rest."* The function that does this is softmax. It exponentiates every score (making them all positive and exaggerating the differences) and then normalizes so the row adds to 1. The outputs are the attention weights.

The whole operation in one line: scale the query–key scores, softmax them into weights, then mix the values.

Mixing the values

The final step is the payoff. Each word takes its row of attention weights and computes a weighted average of every word's value vector. If it put 70% of its weight on trophy, then 70% of trophy's value flows into the new vector for it. This is context mixing made literal: the updated representation of each word is a custom blend of the whole sentence, dialed by relevance.

# self-attention for one sentence (pseudocode, shapes shown)
# x: (seq_len, d_model)  -- one row vector per token
Q = x @ W_q          # (seq_len, d_k)   what each token is looking for
K = x @ W_k          # (seq_len, d_k)   what each token advertises
V = x @ W_v          # (seq_len, d_v)   what each token will hand over

scores  = Q @ K.T            # (seq_len, seq_len)  every query vs every key
scores  = scores / sqrt(d_k) # scale so values stay stable
weights = softmax(scores)    # each row sums to 1

out = weights @ V    # (seq_len, d_v)  context-mixed token vectors

That is self-attention in full. Three projections, a grid of scores, a softmax, a weighted sum. Everything else in a transformer exists to make this operation richer, safer, and stackable — which is exactly what the next guides add.