beam search

Beam search hedges its bets. Instead of committing to one token at each step like greedy decoding, it keeps the top few partial sequences alive at once — say four or eight 'beams' — and extends all of them in parallel, scoring each by the total probability of the whole sequence so far. At the end it returns the highest-scoring complete sequence. It is like exploring several promising trails through a forest simultaneously and only later deciding which one actually led somewhere good.

This often finds higher-probability text than greedy can, which is why it long dominated machine translation and other tasks with one fairly correct answer. But for open-ended generation it backfires: maximizing total probability pushes toward short, safe, repetitive output, because the genuinely likely continuations of human text are not the same as the single most probable string. Beam search is also slower and heavier, since it runs the model on several sequences at once. Modern chat LLMs mostly use sampling instead.