transfer learning

Someone who already plays the piano can pick up the organ far faster than a total beginner — the years spent learning notes, rhythm, and finger control carry over. Transfer learning gives a machine the same head start: instead of training a model from scratch for every new task, you take a model that already learned something on a big, related task and adapt it to your smaller one.

In practice, you start from a model pre-trained on a huge dataset — say, millions of general images or a vast pile of text. Most of what it learned is generic: edges and textures for images, grammar and word meanings for text. You keep that and just retrain the last layers, or gently nudge the whole thing, on your own modest dataset. This step is called fine-tuning. Because the model already understands the basics, it needs far less data and far less compute to become good at your specific job.

Transfer learning is one of the most useful ideas in modern machine learning, and it is the reason a small team can build a strong classifier from a few hundred examples. The honest caveat: it works only when the original task and yours are related enough that the borrowed knowledge actually applies. Transfer a model trained on photographs to medical scans and it may help — or it may carry over assumptions that mislead it. The further apart the two domains, the weaker the transfer, and sometimes it even hurts.