JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

When (and When Not) to Fine-Tune

Fine-tuning is one of three ways to shape an LLM's behavior. Learn what it really does and how to tell when it's the right tool.

What fine-tuning actually is

Fine-tuning means taking a model that has already learned language from a huge corpus and continuing to train it — with a much smaller, much more focused dataset — so its weights shift toward the behavior you want. You are not building a large language model from nothing; you are nudging an existing one. The base model already knows grammar, facts, and reasoning patterns. Fine-tuning teaches it style, format, and task on top of that foundation.

\theta \leftarrow \theta - \eta\,\nabla_{\theta}\,\mathcal{L}(\theta)

Fine-tuning keeps running the very gradient step pretraining used — it just nudges the weights \theta on a much smaller, focused dataset.

Three levers, not one

Before you fine-tune, know your alternatives. There are three main ways to change what an LLM does, from cheapest to most involved — and fine-tuning is the heaviest. The art is in choosing between prompting, RAG, and fine-tuning.

  1. Prompting — change the instructions you send at request time. Prompting costs nothing to set up, ships in minutes, and is where you should almost always start.
  2. Retrieval (RAG) — fetch relevant documents and paste them into the prompt. RAG is the answer when the model lacks knowledge (private docs, fresh facts), not when it lacks behavior.
  3. Fine-tuning — change the weights. Reach for it when prompting and retrieval can't reliably produce the style, format, or skill you need, or when you want a smaller, cheaper model to behave like a bigger one.
Retrieval (RAG) — the middle lever — fetches relevant documents at query time instead of changing any weights.

A retrieval-augmented generation pipeline: a query retrieves documents that are fed to the model as context.

A map of the fine-tuning landscape

"Fine-tuning" is an umbrella over several distinct procedures, which the rest of this track unpacks. The first split is what you train on: continued pretraining feeds more raw text to teach a domain; supervised fine-tuning feeds input–output pairs to teach a task; preference tuning (covered in the alignment track) uses human judgments to shape tone and safety.

The second split is how much you change: a full fine-tune versus a parameter-efficient one. Full fine-tuning updates every weight and needs serious hardware; parameter-efficient methods freeze the model and train a tiny add-on. Guide 3 lives here.

What you'll need

Every fine-tune rests on three pillars. Weakness in any one shows up as a disappointing model, so be honest about all three before you start.

  1. Data — a set of examples that demonstrate the behavior you want. Quality and consistency matter far more than raw volume.
  2. A base model — usually an open-weight model. Decide up front whether you start from a base model or an already chat-tuned model, because that changes your data format.
  3. An evaluation — a way to measure whether the fine-tune helped, ideally before you ever ship it. Without this you are flying blind.
The third pillar is an evaluation: hold out validation and test data so you can measure whether the fine-tune actually helped.

Data divided into separate train, validation, and test partitions for held-out evaluation.

A quick decision checklist

Run through these questions before committing. If you answer "no" to the first two, you probably don't need to fine-tune yet.

  1. Have I already tried a strong prompt and (if it's a knowledge problem) retrieval, and still fall short?
  2. Can I describe the desired behavior with consistent examples — at least a few hundred?
  3. Is the problem about behavior, style, or format (good for fine-tuning) rather than missing facts (better for RAG)?
  4. Do I have an evaluation that will catch regressions, not just cherry-picked wins?