activation steering

Once you have found the internal direction that stands for a concept — the model's 'feature' for, say, honesty, or anger, or talking about weddings — a tempting question follows: what if you just turn that dial up or down while the model is running? Activation steering does exactly that. Instead of changing the prompt or retraining, you reach inside as the model computes and add a chosen direction to its activations, nudging its behavior in real time.

Concretely, you first obtain a steering vector for the trait, often by taking the difference between the model's average activations on examples that have the trait and examples that lack it (for instance, kind replies minus rude replies). Then, at some layer during generation, you add a multiple of that vector to the running activations. Crank a 'refusal' direction up and the model refuses more; push it down and it complies more; add a 'Golden Gate Bridge' vector and the model starts mentioning the bridge unprompted. The size and layer of the nudge are knobs you tune.

This matters two ways for safety. As a research probe, steering is strong causal evidence that a direction really carries a concept: if adding it reliably changes behavior in the predicted way, the feature is doing real work, not just correlating. As a control method, it hints at a lightweight way to make models safer (steer away from deception or toxicity) without expensive retraining. But the honest caveats are large: steering is blunt, often degrades fluency or has side effects on unrelated abilities, can be undone or overridden, and works on the behavior we can name — it is a promising lever, not a guarantee of control.