fine-tuning
/ FYNE-too-ning /
Fine-tuning is taking a model that already knows a lot in general and giving it a focused, much smaller course of training so it gets good at one particular thing. Picture a well-read graduate who can write about almost any subject; fine-tuning is the few weeks of on-the-job training that turns her into a competent paralegal. The broad knowledge was already there; you are nudging it toward a narrower groove.
Technically, you start from a pretrained model and continue training it on a curated dataset — say, a few thousand examples of medical questions and good answers. The model's internal numbers (its weights) shift a little to fit the new examples. Because it already learned language during pretraining, it needs far less data and computing power than starting from scratch. Variants like LoRA make this cheaper still by adjusting only a small slice of the model rather than all of it.
Two honest caveats. First, fine-tuning can cause "catastrophic forgetting": push hard on the new task and the model can lose some of its old general skill. Second, fine-tuning teaches style and patterns far more reliably than it injects fresh facts — if you want the model to use specific, up-to-date knowledge, retrieval-augmented generation (feeding it documents at question time) is often safer than trying to bake the facts into the weights.
A bank starts from a general language model and fine-tunes it on 5,000 past customer-service chats. Afterward it answers in the bank's tone, knows the product names, and follows house policy — without anyone retraining the model from scratch, which would cost a thousand times more.
A small, focused course turns a generalist into a specialist.
A common mistake is reaching for fine-tuning when you really need retrieval. If the problem is "the model doesn't know our latest prices," feeding it the price list at question time usually beats fine-tuning; fine-tuning shines for changing how the model behaves, not for storing facts that keep changing.