JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

Choosing the Architecture: Prompting vs RAG vs Fine-Tuning, and Orchestration

Three levers move an LLM toward your task. Most teams reach for the wrong one first — here is how to choose.

The three levers

When a base model does not do what you want, you have three levers — and choosing among prompting, RAG, and fine-tuning is the most consequential design decision in the whole project. Prompting changes the instructions you send. RAG — retrieval-augmented generation — feeds the model relevant facts at request time. Fine-tuning changes the model's weights with further training. They are not rivals; mature products often use all three. The mistake is reaching for the heaviest lever first.

Retrieval-augmented generation: the RAG lever fixes what the model doesn't know by fetching the right facts at query time.

A pipeline where a user query retrieves documents that are added to the prompt before the model generates an answer.

A decision framework

Work from cheap-and-fast to expensive-and-slow, and stop as soon as the task is solved well enough.

  1. Start with a strong prompt and the best general model. Surprisingly often, this is the whole answer.
  2. If failures come from missing or stale facts, add RAG: chunk your documents, embed them into a vector store, retrieve the top matches, and inject them into the prompt.
  3. If the task is multi-step or branches, decompose it with prompt chaining before you reach for anything heavier.
  4. Only fine-tune when prompting plus RAG plateau: you need a consistent house style, a rigid output format, lower per-call cost, or a smaller model to match a big one on your narrow task.
Fine-tuning the last resort: a reward model trained on human preferences tunes the policy once prompting plus RAG plateau.

Human preferences train a reward model that then guides policy tuning of the language model.

Orchestration: gluing the steps together

Once an app has more than one model call — retrieve, then answer; classify, then route; plan, then act — you need orchestration. An orchestration framework manages the control flow: it sequences calls, passes state between them, wires in retrieval and tools, handles retries, and gives you tracing across the whole chain. You can hand-roll this, and for simple apps you should; frameworks earn their keep when the graph of steps gets branchy, when you want swappable components, or when you need observability out of the box.

Orchestration as a loop: the model reasons, acts (a tool call or step), and observes the result before deciding the next move.

A loop of reason, act, and observe steps driving a multi-step LLM application.

Common composition patterns

Orchestrated apps tend to take a few shapes. A chain runs fixed steps in order. A router uses a cheap classification call to pick which branch (or which model) handles a request. A map-reduce splits a huge document, summarizes the pieces in parallel, then merges them. And an agent hands control to the model itself: it decides which tool to call next, loops, and stops when done. Chains and routers are predictable and easy to evaluate; agents are powerful but harder to make reliable — which is exactly why the last guide in this track treats agentic products as the frontier.