supervised fine-tuning (SFT)

A pretrained base model is a fluent next-word predictor with no particular inclination to follow instructions — ask it a question and it might continue with more questions. Supervised fine-tuning is the step that teaches it to behave like an assistant. You assemble a curated set of input-output demonstrations — a prompt and the ideal response a good helper would give — and continue training the model to reproduce those responses, so it learns the shape of helpful, instruction-following behavior.

Mechanically it is the same maximum-likelihood objective as pretraining: minimize the cross-entropy of predicting each next token, but now over the demonstration completions. Two engineering details matter. The loss is usually masked so it only counts the assistant's tokens, not the prompt, so the model is graded on what it should generate rather than on copying the input. And inputs are wrapped in a chat template with explicit role markers, which the model learns as the scaffolding that separates user turns from its own.

SFT is the workhorse first stage of post-training and often the foundation that later preference methods refine. Its ceiling is set by demonstration quality and coverage: the model imitates exactly what it is shown, so gaps and biases in the data become gaps and biases in behavior, and pure SFT cannot easily express what to avoid — only what to do — which is precisely why an alignment stage usually follows.