dense vs sparse retrieval

Retrieval comes in two broad families. Dense retrieval uses learned embeddings — "dense" because the vectors are short and every dimension carries some meaning. Sparse retrieval, with BM25 as the classic example, scores documents by overlapping keywords; its vectors are huge and mostly zero, one slot per vocabulary word, lit up only where a word actually appears. One thinks in meanings, the other in literal terms.

Each has a characteristic blind spot. Dense retrieval captures meaning, synonyms, and paraphrase, but can fumble exact terms, rare proper names, and specific numbers. Sparse retrieval nails exact-keyword and code-like matches but is blind to rewording — search "car" and it will never find a passage that only says "automobile". Deciding which failure you can live with for a given corpus guides the choice, and often pushes people to use both together.