shadow deployment

Before you trust a new model with real decisions, you want to see how it behaves on real traffic without anyone being affected by its mistakes. Shadow deployment does exactly that: every live request is mirrored to the new model in parallel with the current production model, the new model computes its prediction, and that prediction is logged and discarded. Users only ever experience the incumbent's output. The challenger runs in the dark, exercised by the full production distribution, judged on data it never saw at evaluation time.

Operationally, the serving layer duplicates the request, fans it out to both models, and returns only the production response to the caller while recording the shadow response, its latency, resource usage, and any errors. Because the shadow model touches production inputs, this surfaces the failures that offline evaluation cannot: feature-pipeline mismatches, schema drift, tail-latency blowups, and crashes on rare inputs. Comparing shadow predictions against eventual ground-truth labels (when they arrive) gives an honest, no-risk read on real-world accuracy.

The cost is that you pay for inference twice and must guarantee the shadow path has no side effects, no writes, no external calls, no quota consumption, or the dark launch stops being dark. Shadow mode also cannot measure outcomes that depend on the model actually acting, such as how users respond to its recommendations; for that you eventually need a canary that serves real traffic.