long-tail data handling

Real-world data follows a long-tailed distribution: a few head classes have abundant examples while a long tail of rare classes have only a handful each. A model trained naively becomes excellent at the common cases and nearly blind to the rare ones — yet the rare cases (an unusual disease, a fraud pattern, an edge-case object) are often exactly the ones that matter. Long-tail handling is the set of techniques that recover decent accuracy on the tail without wrecking the head.

The interventions sit at three levels. Data-level methods resample: oversample tail classes (and synthesize variants), undersample the head, or augment the tail. Loss-level methods reweight: class-balanced loss using the effective number of samples, logit-adjustment that shifts decision boundaries by the log prior, and margin losses (LDAM) that enforce larger margins for rarer classes. Representation-level methods decouple feature learning from the classifier — learn features on the natural distribution, then rebalance only the final classifier — since the backbone often learns fine while the classifier is what skews toward the head. For foundation models, retrieval and few-shot transfer let tail knowledge ride on broad pretraining.

The honest tension is the head–tail trade-off: aggressive rebalancing lifts rare-class recall but can degrade overall accuracy and calibration, so the right operating point depends on the real cost of each error type.