retrieval-augmented fine-tuning (RAFT)

A model used inside a retrieval pipeline is handed documents at inference time, but if it was never trained to live in that setting it tends to either ignore the evidence or be derailed by irrelevant passages. Retrieval-augmented fine-tuning trains the model for the open-book exam it will actually sit: every training example presents a question alongside retrieved passages, deliberately mixing genuinely relevant documents with plausible-looking distractors, and the model learns to answer by leaning on the right ones.

The teaching signal is what makes it work. Targets are written as chain-of-thought answers that quote and cite the supporting passages verbatim, so the model is rewarded for grounding each claim rather than reciting parametric memory. A fraction of examples include only distractors, forcing the model to learn when the retrieved context does not contain the answer. The effect is a model that reads its context critically: it cites what helps and discounts what does not, which is exactly the skill a deployed retrieval system needs.

Compared with plain retrieval-augmented generation bolted onto an off-the-shelf model, RAFT yields markedly better robustness to imperfect retrieval and stronger grounding on a target domain. The cost is building supervised data with realistic distractor mixes, and a model tuned this way is specialized to the retrieval style it was trained against.