SAE feature steering

Once a sparse autoencoder has decoded a layer into named features, you can do more than read them — you can turn them up or down and watch the model change. Steering means intervening on a single SAE feature during generation: clamp the feature for, say, the Golden Gate Bridge to a high value, and the model starts working that bridge into whatever it writes. It is a volume knob on a specific concept the model itself learned.

Concretely, you add the feature's decoder direction back into the residual stream with an inflated coefficient, then let the forward pass continue. Because the direction was discovered by the SAE rather than guessed by hand, steering doubles as a causal test of the feature's meaning: if amplifying it produces exactly the behavior its label predicts, that is strong evidence the feature really represents that concept and the model uses it downstream.

Steering is both a research probe and a glimpse of fine-grained control without retraining. Its limits matter: pushing a feature too hard degrades coherence, features interact so one knob can move others, and the catalog of features is incomplete, so you can only steer what the dictionary happens to have isolated.