champion-challenger

Models do not stay best forever; the world drifts and better candidates appear. Champion-challenger is the operating pattern that keeps deployment honest over time: one model is the reigning champion serving most or all decisions, and one or more challengers run continuously on production traffic, scored on the same live metrics. A challenger is promoted to champion only when it durably beats the incumbent on the metric that matters, and the cycle repeats. It is less a single deployment and more a standing tournament.

Mechanically, challengers can be evaluated by shadowing (logging-only, no risk) or by a small served slice (a canary or holdout for unbiased outcome measurement), with the comparison run as a controlled experiment so the win is causal and not a fluke of seasonality or selection. The framework typically automates candidate registration, traffic allocation, statistical comparison against the champion, and a promotion decision, which makes the whole production stack a closed loop in which improvement is continuous rather than a once-a-quarter manual swap.

The discipline it enforces is choosing the right objective and guarding against metric gaming. A challenger that wins on a narrow offline proxy may lose on the business outcome or violate a fairness or latency guardrail. Mature champion-challenger setups evaluate on the production north-star metric with explicit guardrails, and treat a promotion as reversible, since the new champion can itself be challenged and rolled back.