data quality metrics

If a model is only as good as the data it eats, then data quality metrics are the lab tests you run on the food before serving it. Instead of staring at a million rows and hoping, you compute a handful of numbers that say how noisy the labels are, how much of the input space the examples actually cover, and how often two near-identical inputs get contradictory labels. These numbers turn a vague feeling that a dataset is messy into something you can track, compare across versions, and set a release bar against.

Concretely the family splits into three axes. Label-noise estimates the fraction of examples whose given label disagrees with the true label, often via cross-validated model agreement or annotator inter-rater statistics such as Cohen's or Fleiss' kappa. Coverage measures how well the examples populate the feature manifold and the label distribution: density in embedding space, class balance, and gaps where the model will see inputs at deployment that the training set never illustrated. Consistency measures self-contradiction, the rate at which equivalent or near-duplicate inputs carry different labels, which upper-bounds achievable accuracy.

The honest caveat is that none of these is the target you care about, which is downstream task performance. A dataset can score well on every proxy and still be wrong for your application because it samples the wrong population. Treat quality metrics as a triage and monitoring layer, not a guarantee.