The long-tail problem
A vocabulary has tens of thousands of tokens, and at any step thousands of them carry a tiny but non-zero probability. Individually each is unlikely, but together that long tail holds a surprising amount of probability mass — and most of it is nonsense in context. Pure temperature sampling can still reach into that tail and pull out a wrong word, which is one route to incoherence and degeneration. The fix: truncate the distribution first — throw away the unlikely tail, then sample only from what remains.
The softmax turns logits into a probability over the whole vocabulary — its long tail of tiny-but-nonzero tokens is exactly what truncation trims.
Top-k: keep a fixed number
Top-k sampling is the bluntest cut: sort the tokens by probability, keep the top k (say k = 40), throw the rest away, renormalize the survivors back to sum 1, then sample. It guarantees you never pick beyond the k best candidates. Its weakness is rigidity: k = 40 is far too many when the model is almost certain (the right token has 99% probability and 39 distractions tag along) and too few when the model is genuinely torn among many good options.
Top-p (nucleus): keep a fixed mass
Nucleus sampling, better known as top-p, fixes top-k's rigidity by trimming on probability mass instead of count. Sort the tokens, then keep the smallest set whose cumulative probability first reaches p (say p = 0.9) — the nucleus — and sample from it. When the model is confident, the nucleus might be just 1–2 tokens; when it is uncertain, the nucleus naturally widens to dozens. This adaptivity is why top-p (often with a small top-k as a backstop) is the default in most chat systems.
Nucleus sampling keeps the smallest set of tokens whose probabilities already sum to at least p, then renormalizes.
Min-p: a relative floor
Min-p sampling takes yet another angle: it sets the cutoff relative to the top token. Pick a fraction (say min_p = 0.1) and discard every token whose probability is below that fraction of the most-likely token's probability. If the best token sits at 0.6, the floor is 0.06; if the best token is only 0.2, the floor is 0.02 — so the surviving set automatically tightens when the model is confident and relaxes when it is not. Many people find min-p stays coherent even at high temperatures, making it popular for creative writing.
Min-p sets a relative floor: keep a token only if its probability is at least min_p times the top token's probability.
How they stack with temperature
These methods are not rivals — you usually layer them. A common pipeline runs the filters first to define a sane candidate pool, applies temperature to reshape what is left, then draws. The order matters because temperature changes the probabilities the truncation thresholds are measured against.
- Start from the full softmax distribution over the vocabulary.
- Apply a truncation filter (top-k, top-p, or min-p) to keep a sensible candidate set.
- Apply temperature to reshape the surviving probabilities, then renormalize.
- Sample one token from the renormalized set and continue the loop.
Penalizing repetition directly
Truncation curbs garbage, but you can also attack repetition head-on. A repetition penalty lowers the logit of any token that has already appeared, discouraging loops. Close cousins are the frequency penalty (scaled by how many times a token occurred) and the presence penalty (a flat hit the first time a token shows up, nudging the model toward new vocabulary). Use them gently — crank them too high and the model starts dodging words it genuinely needs, like a common article or a person's name.