temperature sampling

Temperature is the single most famous creativity dial. Before the model's raw scores (logits) become probabilities through the softmax, they are divided by a number T. A low temperature (say 0.2) sharpens the distribution so the top tokens dominate and output is focused and predictable; a high temperature (say 1.2) flattens it so unlikely tokens get a real chance, and the text grows more varied — and more error-prone. At T = 1 the distribution is left as the model intended.

The name borrows from physics, where high temperature means more random motion. Setting T = 0 collapses sampling back to greedy decoding: the sharpest possible distribution always picks the top token. The right value depends on the job — near zero for code, math, or extraction where you want one correct answer; higher for brainstorming, fiction, or dialogue where surprise is welcome. Temperature is usually combined with top-k or top-p so that flattening cannot resurrect truly absurd tokens.