causal masking

An LLM is trained to predict the next word, so it must never peek ahead. Causal masking enforces that rule. When the model is figuring out the word at position five, masking blocks it from attending to positions six, seven, and beyond — it may only look at itself and what came before. It is like reading a sentence with a card that covers everything to the right of your finger.

The mask is applied to the attention scores: every entry where a token would attend to a later token is set to negative infinity before softmax. After normalization those entries become exactly zero weight, so no information from the future leaks backward. This single trick lets the model compute predictions for all positions at once during training, while still respecting left-to-right order at every position.

Causal, or autoregressive, masking is what makes a decoder-only model. Encoder models like BERT drop the mask so tokens can see both directions, which is great for understanding but not for generation. The mask is also why an LLM can cache past keys and values: earlier tokens never need recomputing, because nothing later can change what they were allowed to see.