inference-compute scaling

Until recently, almost all the compute went into training; at answer time the model did one quick forward pass and spoke. Inference-compute scaling flips attention to the other end: let the model spend more effort while answering — think longer, draft several attempts, check its own work — and the quality of the answer climbs. It is the difference between blurting the first thing that comes to mind and actually working a problem out.

Concrete forms include long chain-of-thought reasoning, sampling many candidate solutions and voting or picking the best, and search procedures that expand and prune a tree of steps. Empirically, accuracy on hard reasoning tasks rises smoothly as you pour in more test-time compute — its own scaling curve, separate from the training laws. A modest model allowed to think can match a much larger model answering instantly, which reshapes where the compute is worth spending.

The gains are real but bounded and uneven: extra thinking helps on problems with checkable structure far more than on open-ended ones, and it trades latency and cost on every single query, not once at training.