debate (AI safety)

Debate is a scalable-oversight protocol in which two capable models argue opposing answers to a question and a weaker judge, often a human, decides who won. The bet is asymmetric: it is easier to point out a flaw in an argument than to construct a flawless one, so if telling the truth has a debating advantage, a limited judge can be steered toward correct answers on questions they could never verify unaided. Training rewards each debater for convincing the judge.

Formally it is a two-player zero-sum game whose equilibrium is meant to favor honesty, analyzed in complexity terms as letting a polynomial-time judge adjudicate problems a single untrusted model could not be trusted to solve directly. A debater can quote sources, recursively decompose the dispute, and challenge any specific claim the opponent makes, forcing lies to defend themselves at the point of attack where the judge can focus their limited scrutiny.

Empirically debate helps on some tasks and not others, and it has known failure modes: an obfuscated argument that is true but too complex to rebut in the available steps can beat a simpler true one, persuasion can drift from correctness toward rhetoric, and a judge with systematic biases can be exploited by both debaters at once. It is a promising but unproven mechanism for supervising superhuman systems.