ML test score

Traditional software has well-worn checklists for what production-ready means, but an ML system can pass all of them and still be a liability, because its quality lives in data and behavior the code does not express. The ML Test Score, introduced in a Google paper on the rubric for ML production readiness, is a structured scorecard that asks pointed questions across the dimensions where ML systems silently rot: data, features, model development, infrastructure, and monitoring. Each question earns points for being both implemented and automated, and the lowest-scoring category caps your overall maturity.

The rubric groups tests into four areas. Data and feature tests check schemas, distributions, feature importance, and privacy. Model development tests cover reproducibility, offline-to-online metric correspondence, and guards against regressions. ML infrastructure tests cover training reproducibility, integration of the full pipeline, and the ability to roll back. Monitoring tests check for training-serving skew, drift, staleness, and numerical stability in production. Scoring each as manual, automated, or absent yields a number that makes technical debt visible and comparable across teams.

Its value is less the exact score than the conversation it forces: a team that has a high-accuracy model but cannot reproduce its training, cannot detect skew, and cannot roll back is, by this rubric, not production-ready however good the offline metric looks. The score reframes reliability as a property of the whole system and its tests, not of the model's accuracy alone.