open-set / out-of-distribution classification

Standard classifiers operate under a closed-world assumption: every test image is assumed to belong to one of the C training classes, so the model is forced to assign one of them no matter what. Open-set recognition removes that assumption. The model may encounter inputs from classes it has never seen, or images that are not the right kind of thing at all, and the correct behaviour is to flag them as 'unknown' or 'none of the above' rather than confidently misclassifying them as a known class.

This matters because deployed systems inevitably meet the unexpected, and overconfident deep networks are notorious for assigning high probability to garbage or out-of-distribution inputs. A medical classifier should not label an unfamiliar pathology as the nearest known disease with 99 percent confidence; a self-driving perception stack should not silently force every novel object into a training category. Open-set recognition adds the ability to say 'I do not know'.

Practically, it is implemented by computing a confidence or novelty score and rejecting inputs that fall below a threshold. Scores range from the simple, such as maximum softmax probability or its entropy and temperature-scaled energy scores, to the more elaborate, such as distance to class prototypes in feature space, density models, or methods like OpenMax. Evaluation differs from closed-set accuracy: you measure how well known and unknown inputs are separated, typically with AUROC, alongside accuracy on the knowns. It is closely related to out-of-distribution detection and to the reject option in classification.