mesa-optimization
/ mesa: MAY-suh /
When you run a search for something, you can sometimes end up building a thing that searches on its own. Evolution is the clearest analogy: natural selection is an optimization process searching for genes that reproduce, and one thing it produced was humans, who are themselves planners and optimizers pursuing goals (love, art, ice cream) that are not literally maximize reproduction. The outer search produced an inner searcher with its own aims.
Mesa-optimization is that pattern inside machine learning. A base optimizer, such as gradient descent, searches over models to find one that scores well. If the model it finds is itself running some kind of search or optimization over its options toward an objective, then that model is a mesa-optimizer and the overall phenomenon is mesa-optimization. The prefix mesa is Greek for within or below, deliberately the opposite of meta. A toy case would be a trained game-playing network that, inside its forward pass, effectively searches several moves ahead rather than just reacting.
Why care? Because the mesa-optimizer is optimizing for its own internal goal, the mesa-objective, which need not equal the objective we trained for. That gap is precisely the inner-alignment problem and the doorway to deceptive alignment. Be honest about the status: mesa-optimization comes from a 2019 paper (Risks from Learned Optimization by Hubinger and colleagues) and is still largely a theoretical framework. Whether, and exactly when, today's networks contain genuine internal optimizers rather than bundles of learned heuristics is an open and debated question.
Evolution (a base optimizer) selected for reproduction yet produced humans, who optimize for their own goals like art and sugar; in the same way, gradient descent could in principle produce a model that runs its own internal optimization.
An optimizer producing an optimizer, with goals that can diverge.
Mesa-optimization is a hypothesis about how trained models might work, not an established fact about current systems; many networks may rely on learned heuristics rather than a true internal search.