repetition penalty

Left alone, LLMs love a rut: once a phrase appears, the model often finds it the most probable continuation, and you get 'the the the' or a paragraph that keeps circling the same sentence. A repetition penalty fights this by lowering the scores of tokens that have already appeared, before the next token is chosen. Related variants — a frequency penalty that grows with how often a token was used, and a presence penalty that fires once a token shows up at all — give finer control over how harshly recurrence is discouraged.

The mechanism is a nudge to the logits: divide or subtract from the score of any token in the recent context so it becomes less attractive. Used gently it keeps text fresh; cranked too high it backfires, since some repetition is correct — a name, a keyword, code syntax, the word 'the' itself. Tuned poorly, the model dodges natural words and the prose turns strange. It treats a symptom of degeneration rather than its root cause, so it pairs best with sensible sampling.