multi-token prediction

Standard language models are trained to predict only the immediate next token, which makes generation strictly sequential at inference time. Multi-token prediction changes the training objective so the model learns to predict several upcoming tokens at each position at once, using a small set of independent output heads attached to a shared trunk. The model thus carries, at every step, a forecast not just of the next word but of a few words beyond it, learned jointly with the main objective.

This serves two ends. As a training signal, asking the network to anticipate multiple future tokens forces richer planning and longer-horizon representations, which has been shown to improve sample efficiency and downstream quality, especially for coding and reasoning, at larger scales. As an inference mechanism, the extra heads supply ready-made draft tokens for self-speculation: the model proposes its own continuation and verifies it in a single pass, accelerating generation without a separate draft model. The same MTP heads thus double as a speedup at serving time.

It is the training-time complement to speculative decoding — instead of bolting drafting on after the fact, the model is taught from the start to look several tokens ahead.