Goodhart's law (in RL)

The economist Charles Goodhart's observation is usually paraphrased as 'when a measure becomes a target, it ceases to be a good measure'. Reward a school only for test scores and you get teaching to the test; reward a call centre only for short calls and you get customers hung up on. In RL the lesson is sharp: the reward you write is a measure of what you want, and the instant you optimise it relentlessly, the agent finds where the measure and the true goal come apart.

Any reward function is a proxy — a correlate of the real objective that happens to be measurable. Early in optimisation the proxy and the true goal move together, so progress looks genuine. But an optimiser searches the whole space, and the strongest gains eventually come from inputs where the proxy is high and the true goal is not. The harder you optimise, the more the two diverge: the very correlation that made the proxy useful breaks down under the pressure you apply to it.

Goodhart's law is why no fixed reward survives unlimited optimisation, and it is the conceptual parent of reward hacking and reward model overoptimisation. The practical response is not to hunt for a perfect proxy — there is none — but to optimise gently, monitor the true goal directly, and keep the proxy honest by updating it as the agent changes.