document chunking

You cannot drop a three-hundred-page manual into a prompt and you would not want to — most of it is irrelevant to any single question. Chunking is the step that splits long source texts into bite-sized passages, perhaps a few sentences or a single paragraph each, so the retriever can fetch just the slice that matters and leave the rest behind.

Chunk size is a balancing act. Make chunks too large and a passage covers several topics at once, diluting its relevance score and wasting precious context budget; make them too small and a single coherent idea gets scattered across separate chunks, so no one passage tells the whole story. Common strategies split on paragraphs, on section headings, or at a fixed token count. Each finished chunk is then embedded and stored on its own, becoming the atomic unit that retrieval returns.