machine unlearning

Machine unlearning is the task of removing the influence of specific training data, or a specific capability, from an already-trained model, so the result behaves as if that data had never been seen, without paying the cost of retraining from scratch. The motivating cases are concrete: honoring a deletion request under privacy law, scrubbing copyrighted or toxic text, or stripping a dangerous skill such as detailed bioweapon synthesis from a model that otherwise stays useful.

The gold standard is exact unlearning, retraining on the dataset minus the forgotten points, which is correct but usually too expensive, so the field develops approximate unlearning: gradient-ascent or fine-tuning updates that push the model away from the forget set while a retain set preserves general ability, often with a certified bound on how close the result is to the exactly retrained model. Capability unlearning instead targets a concept, degrading performance on a hazardous domain while leaving benign behavior intact.

The hard part is verification and durability. A model can appear to have forgotten while the knowledge survives latently and is recoverable by a few fine-tuning steps, a jailbreak, or an indirect prompt, so suppression is not the same as removal. Unlearning also trades off against utility and can damage entangled abilities, and evaluating it honestly requires adversarial relearning attacks, not just checking that the model now refuses or answers wrongly.