lost in the middle

Even when a fact fits comfortably inside the context window, where it sits matters. Researchers found a U-shaped pattern: models use information placed near the beginning or the very end of a long input far more reliably than the same information buried in the middle. Tuck the one crucial sentence halfway through twenty pages and the model may answer as if it never saw it, while the same sentence at the top or bottom is picked up cleanly.

This is a quiet, counterintuitive failure, because the data is right there — nothing was truncated, yet attention simply favors the edges. It bites hardest in retrieval and long-document tasks, where the passage that answers the question might land in the weak middle zone. Practical defenses are to put the most important material at the start or end, keep inputs only as long as they need to be, and re-rank retrieved passages so the strongest evidence sits where the model actually looks.