scalable oversight

It is easy to grade a child's arithmetic, because you can check every answer yourself. It is much harder to grade a doctoral thesis in a field you have never studied. As AI systems start doing tasks that are too large, too specialized, or too numerous for a human to check directly, we hit a wall: how do we keep giving them correct feedback and rewards when we can no longer reliably judge their work? Scalable oversight is the name for that whole challenge.

Precisely, scalable oversight is the umbrella problem of providing a trustworthy training signal and evaluation for tasks that unaided humans cannot fully judge. The leading idea across proposed methods is to use AI to help humans supervise AI: iterated amplification breaks hard questions into checkable pieces; AI safety via debate pits two AIs against each other for a human judge; recursive reward modeling trains helper models that make hard outputs evaluable; and weak-to-strong generalization studies whether weak supervisors can still elicit good behavior from stronger models. Researchers test these with sandwiching experiments, where a task is hard for a non-expert human but checkable by experts, to see if a method lets the non-expert supervise well.

It matters because outer alignment lives or dies on the quality of the training signal. Without scalable oversight, a reward model for a superhuman task becomes an unreliable proxy that the system can quietly exploit, the Goodhart trap again. The honest status: this is an active research agenda, arguably the central practical bottleneck for aligning very capable systems, and no proposed method is yet known to fully solve it.