nucleus sampling (top-p)

Nucleus sampling, better known as top-p, fixes top-k's blind spot by sizing the candidate pool to the model's own confidence. You pick a probability mass p — commonly 0.9 — and the method gathers tokens from most likely downward until their probabilities first add up to at least p. That set is the 'nucleus'; everything in the long tail is discarded, the nucleus is renormalized, and the next token is sampled from it.

The beauty is that the shortlist breathes with context. When the model is confident, two tokens may already cover ninety percent of the mass, so the nucleus is tiny and the output stays on track. When it is uncertain, dozens of tokens are needed to reach p, so the nucleus widens and creativity opens up. Top-p around 0.9 to 0.95 with a moderate temperature is the everyday default for chat and creative writing, and is often layered on top of a temperature setting.