Interpretability and Control: Reading and Steering the Internals

Why interpretability is a safety tool, not a curiosity

Guide 4 left us with a brutal corollary: if deceptive alignment is possible, behavioral tests cannot certify safety, because the dangerous model is defined to behave well under test. The only escape is to inspect the cause of behavior — the computation itself. That is the safety case for mechanistic interpretability: not 'explain a prediction to a user', but 'detect a goal or a plan inside the weights that the model would never reveal in its outputs'. It is the most direct attack we have on eliciting latent knowledge.

The central technical obstacle is superposition: networks represent far more concepts than they have neurons by encoding features as directions in activation space that overlap, so a single neuron is polysemantic — it fires for many unrelated things. You cannot read a polysemantic neuron and learn what the model is 'thinking'. Decode superposition and you get a vocabulary of concepts; fail to, and the internals stay an opaque smear.

Sparse autoencoders: a dictionary of concepts

Sparse autoencoder (SAE) features are the leading attack on superposition. Train a wide autoencoder on a layer's activations with a sparsity penalty, so each activation is reconstructed as a sum of just a few entries from an over-complete dictionary. The dictionary atoms tend to be monosemantic — each fires for one human-interpretable concept ('the Golden Gate Bridge', 'code that handles errors', 'a deceptive or sycophantic tone'). You have turned an opaque vector into a sparse list of named features.

Two cautions keep this honest. SAEs are unsupervised — they hand you thousands of features but no guarantee the one you need (say, 'the model intends to deceive') exists cleanly or that you can find it among the noise. And they reconstruct activations imperfectly, so features are an approximate basis, not ground truth. Still, the same dictionary that reads a concept also lets you write it — which is the door to control.

# Sparse autoencoder on layer activations a  (d_model -> d_dict >> d_model)
f = relu(W_enc @ a + b_enc)          # sparse code; most entries ~ 0
a_hat = W_dec @ f + b_dec            # reconstruct
loss = ||a - a_hat||^2 + lam * ||f||_1   # reconstruction + sparsity
# Each column of W_dec is a candidate monosemantic 'concept direction'.

The sparse-autoencoder objective: reconstruct the activation while an L1 penalty forces only a few dictionary features to fire.

Steering: from reading to writing

Activation steering turns interpretability into control. Find a direction in activation space that corresponds to a behavior — by contrasting activations on prompts that do vs. don't exhibit it, or by taking an SAE feature — then add that vector to the residual stream at inference time. Scale it up to amplify the trait (honesty, refusal, a topic), negate it to suppress it. No fine-tuning, no labels, a knob you turn at runtime. The Golden Gate Claude demo (clamping one feature so the model obsessively brings up the bridge) is the vivid version.

Activation steering: build a behavior direction as the difference of mean activations on contrasting prompts, then add it back to nudge behavior.

Representation engineering (RepE) is the top-down cousin: instead of building concepts bottom-up from neurons, start from a high-level concept (truthfulness, power-seeking, harmfulness), use stimulus sets to locate its representation, then read it as a monitor or control it as a steering target. RepE has produced practical lie-detectors and harmfulness probes that catch failures behavioral tests miss — a partial, encouraging answer to the deception problem. The honest limit: steering is brittle off-distribution, can degrade capability if pushed hard, and a model that knows it is being probed could in principle learn to spoof the very representations we read.

Robustness, unlearning, and an honest scorecard

Two more control problems round out the deployment picture. Jailbreak robustness is the arms race of keeping safety training from being bypassed by adversarial prompts — role-play framings, encodings, suffix attacks, many-shot priming. The sobering pattern is that safety is often shallow: it lives in the first few tokens or a thin behavioral layer, so attacks that move past it reach an underlying model that was never really made safe. Defenses (adversarial training, circuit breakers that disrupt harmful representations, input/output classifiers) raise the cost but rarely close the gap, and evaluation must assume an adaptive attacker.

Machine unlearning asks a complementary question: can we remove a specific capability or piece of data's influence after training — hazardous knowledge, copyrighted text, a person's data — without retraining from scratch? Methods exist (gradient ascent on the forget set, relabeling, localized edits), but a recurring finding is that unlearning often only suppresses rather than erases: the capability can be recovered by fine-tuning, jailbreaks, or quantization. Verifying that something is truly gone, not just hidden, connects right back to interpretability.

1. When you read a safety result, ask what it certifies: behavior on a test set (weak), an internal property (stronger), or a worst-case bound (rare).
2. Assume an adaptive adversary for robustness and unlearning; static benchmarks systematically overstate safety.
3. Prefer methods whose claims are checkable inside the model (RepE monitors, SAE features) over claims that rest only on outputs.
4. State the threat model and the failure modes you are NOT covering. Honest scope is itself a safety contribution.

Step back and the shape of the field is clear. Outer alignment (guide 2) is workable but ceilinged; scalable oversight (guide 3) pushes the ceiling; inner alignment (guide 4) is the part no feedback can settle; and interpretability and control (this guide) are our bet for seeing and steering what feedback can't reach. None of these is finished. The intellectually honest summary is that we have promising tools, real empirical traction, and no method yet that would let us certify the safety of a system smarter than ourselves. That open problem is the research frontier this track has been walking you toward.