chunk overlap

If you slice a document at hard boundaries, a sentence or an idea can get cut clean in half — the tail of one chunk and the head of the next each hold a fragment that only makes full sense together. Chunk overlap fixes this by letting consecutive chunks share a little text at their edges, so nothing important falls down the crack between two passages.

In practice each chunk repeats the last sentence or two of the chunk before it, which means any idea straddling a boundary appears intact in at least one chunk and can be retrieved as a whole. The overlap is a tunable window: too little and you still split thoughts; too much and you store redundant copies that bloat the index and can return several near-duplicate hits for one query. A modest overlap is a cheap, widely used safeguard.