contrastive decoding

Contrastive decoding picks the next token by comparing two models instead of trusting one. You run a strong 'expert' model and a weaker 'amateur' — often a much smaller version of the same family — on the same context, and favour tokens the expert likes much more than the amateur does. The intuition: failure modes both models share, like bland filler, generic phrasing, or surface repetition, get cancelled out, while the expert's genuine knowledge stands out where it diverges from the amateur.

The payoff is text that is more fluent and less degenerate without simply cranking up probability the way greedy or beam search would. A related variant, contrastive search, instead penalizes a candidate for being too similar to tokens already generated, directly discouraging the loops that plague open-ended generation. Both cost extra computation — a second forward pass or an added similarity term — and need tuning, but they show that what you subtract from a distribution can matter as much as what you keep.