retrieval-augmented generation (RAG)

Picture the difference between a closed-book exam and an open-book one. A plain language model answers from memory alone; retrieval-augmented generation lets it look things up first. Before the model writes a word, the system searches a collection of documents for passages relevant to your question, then pastes those passages into the prompt so the answer is built from text sitting right in front of the model rather than only from what it happened to memorise during training.

A RAG system has two parts: a retriever and a generator. The retriever finds candidate passages — most often by vector similarity to the question — and the best ones are placed in the context window alongside your query. The LLM then generates an answer conditioned on that supplied evidence. Crucially, the model's weights never change; you are steering it with fresh, specific text at the moment of the question, which makes RAG far cheaper and quicker to update than retraining.