mean intersection over union

Mean Intersection over Union is the standard yardstick for semantic segmentation, where the model must label every single pixel with a class (road, car, sky, person). Pixel accuracy alone is a trap here: if 80% of pixels are 'road' and 'sky', a model can score high just by painting big regions correctly while completely missing thin pedestrians. mIoU fixes this by measuring, for each class, how well the predicted region of that class overlaps the true region — and then averaging fairly across classes so small classes count as much as big ones.

For one class, IoU (also called the Jaccard index) is the area of overlap between prediction and ground truth divided by the area of their union: IoU = intersection / union = TP / (TP + FP + FN), counting pixels. The numerator rewards pixels you got right; the denominator penalizes both pixels you wrongly painted this class (FP) and pixels of this class you missed (FN). An IoU of 1 means perfect overlap; 0 means no overlap. Crucially, because the union is in the denominator, you cannot inflate the score by over-painting a region — extra wrong pixels enlarge the union and shrink IoU.

mIoU is then the arithmetic mean of the per-class IoUs over all classes in the dataset. This per-class averaging is the whole point: a class covering 1% of pixels and a class covering 40% contribute equally to the final score, so a model is forced to handle rare and thin structures, not just the dominant background. On benchmarks like Cityscapes, ADE20K, and Pascal VOC, mIoU is the reported number, and it is far more sensitive to small-object and boundary errors than plain pixel accuracy.

Watch the details. IoU is computed by pooling pixels across the whole dataset for each class (the standard 'dataset-level' IoU), not by averaging per-image IoUs, which would unstably reward images that happen to contain a class. Classes absent from an image contribute no pixels but still factor into the mean over classes. For instance and panoptic segmentation, related metrics (mask AP, Panoptic Quality) extend the same overlap idea to distinguish individual object instances. And because IoU is non-differentiable, training typically optimizes a proxy (cross-entropy, Dice loss, or a soft-IoU surrogate) while IoU remains the evaluation target.