residual stream analysis

In a transformer, every layer reads from and writes to one shared vector per token called the residual stream. Think of it as a conveyor belt running through the model: attention heads and feed-forward blocks each add their contribution onto the belt rather than replacing what is there. Residual stream analysis treats this belt as the model's main bus of information and studies what each component deposits and what later components pick up.

Because additions accumulate linearly, you can decompose the stream at any point into the sum of contributions from every earlier head and layer. This makes it possible to ask precise questions: which head wrote the direction that the final unembedding reads to predict this word? Tools like the logit lens, direct logit attribution, and activation patching all lean on this additive structure to attribute an output back to the specific writers that caused it.

The framing reframes a transformer as many small components communicating over a common channel by reading and writing directions. It clarifies a lot, but is partly an idealization: layer normalization rescales the stream nonlinearly, and superposition means a single direction can carry several overlapping messages, so attributions need care.