model monitoring
/ MOD-ul MON-ih-tur-ing /
Model monitoring is the ongoing practice of watching a deployed model in production to catch when it starts misbehaving. A model is not a 'set it and forget it' appliance. The world it was trained on keeps changing, so a model that worked beautifully at launch can quietly decay months later. Monitoring is the dashboard and alarm system that watches for that slow rot, the way a hospital monitor watches a patient's vital signs.
It watches two kinds of things. First, the boring-but-vital operational health: is the service up, how fast is it responding, are errors spiking? Second, and harder, the model's actual quality: are its predictions still accurate, are the inputs it's receiving still the kind it was trained on, has its output pattern shifted? The hard part is that in production you often don't immediately know the right answer — a loan model predicts 'will repay,' but you won't know if that was correct for years — so monitoring leans on proxy signals like input drift and prediction distribution rather than direct accuracy.
Why it matters: most ML failures in the wild are not dramatic crashes but silent degradation — a model that's still running, still returning answers, just increasingly wrong, with no error message to warn anyone. Monitoring is the only thing standing between you and that invisible decay. The honest caveat: it can tell you something looks off, but it rarely tells you why, and it can't fix anything itself. It is a smoke detector, not a firefighter — it buys you the chance to react before a quiet failure becomes a costly one.
A demand-forecasting model runs fine for months, then a new product line floods it with inputs unlike anything it saw in training. Accuracy won't be confirmable for weeks, but monitoring flags the sudden shift in input patterns now — giving the team a chance to retrain before bad forecasts pile up.
When true accuracy is delayed, watching the inputs is the early warning you can actually get.
Monitoring detects problems; it does not diagnose or fix them. An alert that 'something changed' still leaves the hard work — figuring out what and why, and deciding whether to retrain — to people. Don't mistake a green dashboard for a healthy model; mistake it for the absence of the few signals you happened to track.