attention head interpretation

A transformer layer has many attention heads running in parallel, each free to look back at different earlier tokens. Interpreting a head means figuring out its job: which tokens it attends to, why, and what it copies forward when it does. Some heads turn out to do recognizable things — attend to the previous token, point a pronoun at its antecedent, link a word to its part of speech, or find the matching bracket.

Two ingredients describe a head. The attention pattern says where it looks: a query-key comparison that decides, for each token, which earlier positions to pull from. The output-value behavior says what it writes once it has chosen: a value-output transformation that moves some information into the residual stream. Reading both, often across many example sentences and with patching to confirm causality, lets you assign a head a function rather than just a heatmap.

Caution is warranted. A clean attention pattern is suggestive but not proof — where a head looks does not equal what it computes, and many heads are polysemantic or only matter in combination with others. Honest head interpretation pairs the visual pattern with causal interventions to check the head actually does what its picture implies.