canary deployment (ML)

Named for the canary miners carried underground to detect bad air, a canary deployment exposes a new model to a small, bounded slice of real traffic, say one percent of users, while the rest stay on the trusted incumbent. If the canary's live metrics hold up, you widen its share in steps; if they degrade, you cut it off and almost nobody was affected. Unlike a shadow, the canary actually serves its predictions, so it measures real outcomes, not just what the model would have said.

The mechanism is a traffic-splitting router plus automated promotion logic. A fraction of requests is deterministically routed to the canary, typically hashing on a user or session id so a given user has a stable experience, and the system continuously compares canary versus baseline on both system metrics (latency, error rate, saturation) and quality metrics (prediction distribution, business KPIs, and labels as they materialize). Promotion gates advance the rollout only when the canary is statistically non-inferior; otherwise an automated rollback fires.

The hard ML-specific wrinkle is that model quality metrics are often delayed, sparse, or noisy: ground truth may arrive hours or days later, so a tiny canary slice can take a long time to reach significance. Practitioners lean on fast leading indicators (prediction-distribution shifts, calibration, guardrail metrics) for the immediate decision and reserve slower outcome metrics for confirming the full release.