JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

Advanced RAG and Evaluation

Rewrite weak queries, retrieve across multiple hops, let an agent drive the loop — and measure all of it.

Fix the query before you search

Users ask messy questions: vague, full of pronouns, or riding on earlier turns of a conversation (*what about the second one?*). Searching that raw text retrieves poorly. Query rewriting inserts a small LLM step first that rewrites the question into a clean, self-contained search query — resolving it and that, expanding abbreviations, or splitting one tangled question into several.

A close cousin is query expansion: generate a few paraphrases of the question, retrieve for each, and pool the results — useful when the document's wording is hard to guess. Both tricks cost one cheap extra model call and routinely lift retrieval more than any embedding tweak.

Multi-hop retrieval

Some questions can't be answered by one search because the answer depends on an intermediate fact. *Who leads the team that built our billing service?* needs two hops: first find which team owns billing, then find that team's lead. Multi-hop RAG retrieves, reads, forms a follow-up query from what it learned, and retrieves again — chaining searches until it has every piece.

Agentic RAG

Agentic RAG hands the loop to an LLM agent that decides for itself what to do. Instead of a fixed retrieve-then-generate pipeline, the model can choose to search, judge whether the results are good enough, search again with a better query, call other tools (a calculator, a database, a web search), and only answer once it's satisfied. Retrieval becomes one tool among several that the agent invokes as needed.

Agentic RAG hands control to an LLM agent that loops reason → act → observe instead of running a fixed pipeline.

An LLM agent loop: the model reasons about the task, takes an action such as a search, observes the result, and repeats until done.

This flexibility handles questions a fixed pipeline can't — but it costs more calls, runs slower, and is harder to debug because the path varies each time. Reach for agentic RAG when questions are genuinely open-ended; keep the simple pipeline for the common, well-shaped ones. Most production systems mix both.

Evaluating RAG

You can't improve what you don't measure, and RAG evaluation splits cleanly into two halves. Retrieval metrics ask: did the right chunks come back? Build a small set of questions paired with the chunks that truly answer them, then track recall (was the gold chunk retrieved at all) and precision (how much of what came back was relevant).

Retrieval metrics ask whether the right chunks came back — precision and recall over the relevant set.

A Venn diagram of relevant versus retrieved chunks; their intersection defines precision and recall.

Generation metrics ask about the final answer: is it faithful (every claim supported by the retrieved context, no invented facts), relevant (it actually answers the question), and complete? Faithfulness is often scored with an LLM-as-a-judge — a second model checks each sentence against the cited context. Watch both halves: great retrieval with an ungrounded generator, or a faithful generator fed the wrong chunks, both fail.

  1. Build a small but real test set — actual user questions, each with the gold passage and an ideal answer.
  2. Score retrieval and generation separately, so a regression points at the exact stage that broke.
  3. Change one thing at a time — chunk size, embedding model, reranker, prompt — and re-run; keep only changes the numbers reward.
\text{Recall@}k = \frac{\lvert\{\text{relevant}\}\cap\{\text{top-}k\text{ retrieved}\}\rvert}{\lvert\{\text{relevant}\}\rvert}

Recall@k — the share of all relevant chunks that appear in the top-k retrieved results, the core retrieval score to track.