a feature
When you read, your brain seems to track ideas — 'this text is about Paris', 'this sounds sarcastic', 'a quote is open and not yet closed'. A feature is the network's version of one such idea: a particular pattern of internal activity that reliably switches on when a specific human-meaningful concept is present in the input, and stays off otherwise. It is the basic unit of meaning that interpretability tries to read.
Technically, a model represents what it is thinking as a long list of numbers (a vector) at each layer. A feature is usually modelled as a direction in that space: take the dot product of the activity with that direction and you get a number that is large when the concept is present. Researchers have found features that fire for surprisingly clean concepts — the Golden Gate Bridge, code with a security bug, text in a particular language, the tone of flattery — and crucially, nudging the activity along that direction can change the model's behavior accordingly, which is what makes a candidate direction credible as a real feature rather than a coincidence.
Features matter because they are the vocabulary any honest internal audit would need: to ask 'is the model thinking about deception right now?' you first need a trustworthy deception feature. The catch is that features are not handed to us labelled. Individual neurons are usually messy and many-meaning (polysemantic), so the clean features often live as combinations spread across many neurons, which is exactly why tools like sparse autoencoders were built to try to pull them apart.
Researchers isolated a 'Golden Gate Bridge' feature inside a large model. When they artificially amplified it, the model began steering almost every conversation toward the bridge — even describing itself as the bridge — strong evidence that the direction really did encode that concept.
A feature is a direction in activation space that tracks one concept; changing it changes behavior.
A neuron and a feature are not the same thing. Because of superposition, one feature often spans many neurons and one neuron often carries many features.