instruction tuning
/ in-STRUK-shun TOO-ning /
Instruction tuning is the training step that teaches a model to do what you ask instead of merely continuing your text. A raw pretrained model is a master of autocomplete: type "Write a poem about the sea" and it might cheerfully reply "Write a poem about the mountains too," because that is a plausible next sentence. Instruction tuning fixes this by showing the model thousands of examples shaped as request followed by a good response, until it learns that a request is meant to be fulfilled, not echoed.
Mechanically it is a kind of fine-tuning, done on a dataset of instruction-and-answer pairs covering many task types — summarize this, translate that, explain this concept, fix this code. The model learns the general habit of treating the user's words as a command. This single step is much of what turned the unwieldy text-predictors of a few years ago into the helpful assistants people now chat with daily.
Be clear about its limits, though. Instruction tuning makes a model willing and able to follow instructions; it does not make the model correct. A well-instruction-tuned model will confidently follow a request straight into a wrong answer, and it can be talked into following harmful instructions too. Reliability and safety need further work — typically human-feedback methods like RLHF layered on top.
Before: prompted with "List three uses of a paperclip," the raw model writes "List three uses of a stapler" — just plausible-sounding text. After instruction tuning, the same prompt yields an actual numbered list of paperclip uses.
The shift from "continue my text" to "do what I asked."
Instruction tuning (also called supervised fine-tuning, SFT) is usually the first stage of "alignment," followed by preference methods like RLHF or DPO. SFT teaches the model the format of helpfulness; the preference stage refines which of several plausible answers people actually prefer.