reranking retrieved documents

First-stage retrieval is built for speed, so it casts a wide net and hands back, say, the top fifty candidates — some excellent, some only loosely on topic. A reranker takes a slower, more careful second look at that shortlist and reorders it, so the genuinely best passages float to the top before any of them reach the model.

Rerankers are usually cross-encoders: they read the query and one candidate passage together, in the same pass, and output a fine-grained relevance score. That joint reading is far more accurate than comparing two separately made embeddings, but it is too slow to run over a whole corpus. So the pipeline is deliberately two-stage — cheap retrieval narrows millions down to dozens, then an expensive reranker sorts those dozens — and you keep only the top few for the prompt.