Causal Methods: Patching, Circuits, and Faithful Claims

From correlation to cause

Everything so far — probes, attention maps, the logit lens — is correlational: it shows what is present, not what the model uses. To make a causal claim you have to intervene. The simplest intervention is an ablation (消融): zero out or knock out a component and see whether the behaviour breaks. If removing a head destroys the model's ability to close brackets, that head plausibly mattered for closing brackets. Ablation is blunt, but it is the gateway to the sharper tools below.

Activation patching: the workhorse

Activation patching (激活修補) — also called causal tracing — is the field's signature technique. Run the model on a clean prompt and a corrupted one that differs in a single fact, caching activations for both. Then run the corrupted prompt again, but patch in one component's clean activation. If that single substitution flips the answer back to the clean one, that component carries the information that mattered. Sweep the patch across every layer and position and you get a heatmap of exactly where the model stored and moved the deciding fact.

1. Build a clean/corrupted pair that differs only in the thing you're studying (e.g. 'Paris' vs 'Rome').
2. Cache all activations from the clean run.
3. Re-run the corrupted prompt, replacing one component's activation with the cached clean value.
4. Measure how much the output moves toward the clean answer; repeat across components to localize.

Discovering circuits and locating knowledge

Chain many patches together and you can map the whole subgraph responsible for a behaviour — this is circuit discovery (電路發現). The goal is a wiring diagram: these heads detect the subject, those move it to the final position, that feedforward block looks up the attribute, and the unembedding reads it out. Done well, a circuit is a falsifiable claim — you can predict what editing each node will do.

A focused application is concept and knowledge localization (概念與知識定位): finding where a specific fact — 'the Eiffel Tower is in Paris' — lives. Causal tracing repeatedly points at mid-layer feedforward blocks as fact stores, which in turn enables targeted model editing: change the stored association and the model answers 'Rome' consistently, including in paraphrases. Localization plus editing is one of the clearest demonstrations that a mechanistic story can be both true and useful.

Steering, safety, and honest readouts

Causal localization feeds back into control. Activation steering (激活操控) adds a direction — extracted by contrasting activations on 'honest' versus 'deceptive' prompts, for instance — to nudge behaviour at inference. The safety dream is bigger: eliciting latent knowledge (引出潛在知識), reading out what a model internally believes even when its words say otherwise, so we could catch a model that knows it is being deceptive. These are research directions, not solved problems, but they are why interpretability is treated as core safety infrastructure rather than a curiosity.

Activation steering adds a scaled direction v to the hidden state, where v contrasts honest against deceptive activations.