LLM inference

Training is teaching the model; inference is using it. When you type a prompt and the model writes back, that is inference — a forward pass through billions of frozen weights, repeated once for each output token. No learning happens here: the weights never change, and every user request simply reads the same trained network.

Concretely, the model first reads your whole prompt, then produces output one token at a time. Each new token is drawn from a probability distribution over the vocabulary, appended to the running text, and fed back in to predict the next — the autoregressive loop. This is why a long answer costs many forward passes, and why inference, not training, dominates the day-to-day compute bill of a deployed model and is where most serving engineering goes.