catastrophic forgetting

A model that learns something new can clobber what it already knew. Fine-tune hard on medical text and the model may get better at medicine while quietly losing its arithmetic, its coding, or its safety manners. That abrupt loss of prior skills is catastrophic forgetting.

It happens because gradient updates overwrite the shared weights that encoded the old abilities; nothing inherently protects them. The more you train, the larger the learning rate, and the narrower your data, the worse it gets. Mitigations include mixing a slice of general data into the fine-tuning set, using PEFT so most weights stay frozen, keeping the learning rate low, and rehearsal or regularisation that anchors the model near its original.

This is why fine-tuning is a balancing act: you measure both the new skill and the old benchmarks, because gains on one can quietly hide losses on the other.