a sparse autoencoder

Imagine a muddy mixed paint colour and you want to know which pure pigments were stirred together to make it. You cannot un-mix paint, but you could train a helper that, shown the muddy colour, names a short list of pure pigments and their amounts, and proves it got the recipe right by re-creating the exact colour. A sparse autoencoder (SAE) does this for a model's tangled internal activations: it learns to rewrite each one as a short combination of cleaner, more interpretable features.

Mechanically, an SAE is a small neural network with two halves. The encoder takes the model's activation vector and expands it into a much larger list of feature numbers, but with a sparsity rule: only a handful are allowed to be non-zero at a time. The decoder must then reconstruct the original activation from just those few active features. To succeed, the SAE is pushed to discover a dictionary of directions where each one corresponds to a single human-meaningful concept that is usually off — exactly the monosemantic features that superposition had smeared together. When it works, you get readable features like 'this is about the legal system' or 'this code has a bug'.

SAEs have become one of the most exciting interpretability tools because they offer a path from polysemantic neurons to clean features at scale, and large dictionaries of millions of features have been extracted from frontier models. But honesty is essential: the features an SAE finds depend on its settings, some are duplicates or fragments rather than the model's true atoms, an SAE never reconstructs activations perfectly (so it misses some signal), and a labelled feature is a hypothesis to be tested, not a proven part of the model. They are a powerful lens, not a finished theory.