single-label vs multi-label classification

The default classification setup is single-label, also called multi-class: each image belongs to exactly one of C mutually exclusive classes, the model produces one probability distribution that sums to one across classes via softmax, and exactly one label is correct. This is the right framing when the categories genuinely cannot co-occur, such as 'this handwritten digit is a 7'.

Multi-label classification drops the mutual-exclusivity assumption: an image may carry any subset of the labels, so a beach photo can be tagged simultaneously 'sky', 'sea', 'person', and 'sand'. The standard implementation replaces the single softmax with C independent sigmoid units, one per label, each predicting the presence or absence of that concept, and trains them with a per-label binary cross-entropy. There is no longer a single argmax; instead each label is decided independently against a threshold, commonly 0.5.

The distinction changes everything downstream: the loss, the output activation, the decision rule, and the metrics. Plain accuracy is a poor fit for multi-label problems because most labels are absent for any given image, so practitioners instead report per-label precision and recall, mean average precision, or the F1 score, micro- or macro-averaged. Choosing single- versus multi-label is a modelling decision about the world, not a tuning knob.