object detection

Object detection answers two questions at once for every object in an image: where is it, and what is it. The 'where' is a box drawn tightly around the object; the 'what' is a class label like 'dog' or 'car' plus a confidence score. Think of image classification as a librarian who tells you a photo 'contains a dog'; object detection is a librarian who also points to the exact spot and says 'a dog is right here, and a bicycle is over there, and a person is behind them' — for an unknown, varying number of things in one picture.

What makes detection genuinely harder than classification is that the number of outputs is not fixed. A classifier always emits one label vector per image. A detector must emit a variable-length list of (box, class, score) triples — zero objects in an empty sky, dozens in a crowded street. This 'set-valued' output is the source of nearly every design choice in the field: how to enumerate candidate locations, how to decide which candidate matches which ground-truth object, and how to avoid reporting the same object several times.

Concretely, a detector consumes an image and produces, for each detected object, a bounding box (typically four numbers), a class label drawn from a fixed vocabulary plus an implicit 'background' class, and a confidence in [0,1]. Quality is judged by whether predicted boxes overlap the true boxes enough (measured by intersection over union) and whether the labels are right; the standard summary metric is mean average precision (mAP) computed over recall levels and IoU thresholds, as popularized by the PASCAL VOC and COCO benchmarks.

Architecturally the field splits into two-stage detectors (propose candidate regions, then classify and refine them — the R-CNN family), one-stage detectors that predict boxes and classes in a single sweep over the image (YOLO, SSD, RetinaNet), and newer set-prediction transformers (DETR) that cast detection as directly outputting a fixed set and matching it to ground truth. All share the same core sub-problems of candidate generation, label assignment, box regression, and duplicate removal.