fine-tuning vs RAG vs prompting

When a base model does not quite do what you need, you have three broad ways to change its behaviour, in rising order of effort. Prompting steers it with instructions and examples in the context. RAG feeds it the right knowledge at question time by retrieving documents. Fine-tuning actually adjusts its weights by training on examples. Picking among them is one of the most common — and most misunderstood — decisions in LLM engineering.

Prompting is the cheapest and fastest to iterate, changes nothing permanently, and is ideal for shaping format, tone, and simple behaviours. RAG is the right tool when the problem is missing or changing knowledge — private documents, current facts — because it grounds answers in retrievable text without retraining, and its knowledge updates the moment you update the documents. Fine-tuning is for teaching a consistent new skill, style, or format that prompting cannot reliably hit, or for compressing long prompts into the weights; it costs data, compute, and a training pipeline, and it bakes knowledge in at a fixed date.

The common error is reaching for fine-tuning first when the real problem is missing knowledge, which RAG solves more cheaply, or behaviour, which a better prompt often fixes. A useful rule of thumb: prompt until it stops improving, add RAG when the gap is knowledge, and fine-tune when the gap is a behaviour or skill the prompt cannot reach. These are not exclusive — strong systems often combine a fine-tuned model, retrieval, and a careful prompt at once.