fine-tuning
Fine-tuning is the act of taking a model that was already trained on a large general task and continuing to train it on your specific target data, so its already-good features adapt to the peculiarities of your problem. It is the richer half of transfer learning: rather than freezing the pretrained network and only training a new head on top, you let the pretrained weights themselves keep moving. The analogy is hiring an experienced professional and giving them a short, targeted on-the-job training for your particular workplace, rather than teaching a novice everything from scratch — they already know the fundamentals; you are just specializing them.
The central risk is catastrophic forgetting: if you train too aggressively, the large updates overwrite the valuable general knowledge the model arrived with, leaving you no better off than training from scratch. The first line of defense is a small learning rate, typically much smaller than you would use for training from random initialization (often 10× to 100× smaller), so the weights adapt gently without being clobbered. Many practitioners also use a short warmup and a discriminative learning rate that is even smaller for the early, generic layers and larger for the later, task-specific ones, since the early layers need the least change.
Layer freezing is the other main lever and trades adaptation against overfitting and cost. At one extreme you freeze the entire backbone and train only a new output head — fast, data-efficient, and safe when target data is scarce. At the other you unfreeze and fine-tune all layers — most expressive, needed when your domain differs substantially from the source, but data-hungry and prone to overfitting on small sets. A popular middle path is progressive unfreezing: start with the head, then unfreeze layers from the top down as training stabilizes, adapting the most task-specific layers first.
When models are huge, full fine-tuning becomes expensive in both compute and storage (a separate copy of billions of weights per task). Parameter-efficient fine-tuning (PEFT) addresses this by freezing the pretrained weights and training only a tiny number of new parameters: LoRA injects small low-rank update matrices into existing layers, and adapter modules insert small trainable bottlenecks between frozen layers. These reach most of full fine-tuning's quality while training under 1% of the parameters, which is why they have become standard for adapting large vision and vision-language models like CLIP and segment-anything-style models to new tasks.
A two-stage recipe often beats fine-tuning everything at once on small datasets: first freeze the backbone and train only the new head until it stops improving, then unfreeze and fine-tune the whole model at a small learning rate. Fine-tuning a backbone while its randomly-initialized head still emits huge, noisy gradients can corrupt the good pretrained features before the head settles.