mathematical problem solving

Mathematical problem solving covers everything from adding a column of numbers to following a word problem to sketching a proof. It is a favourite testbed for reasoning because answers are often unambiguously right or wrong, which makes progress easy to measure. Modern models, especially reasoning models, have improved dramatically here: they can carry a multi-step word problem, set up the right equations, and handle competition-level questions that stumped earlier systems, usually by writing out their working step by step.

But the surface can mislead, and the limits are instructive. A model can recite a theorem flawlessly yet slip on the plain arithmetic inside it, because digit-by-digit calculation is exactly the kind of precise, mechanical work that next-token prediction does unreliably. That is why pairing the model with a calculator or a code interpreter helps so much: it offloads the exact computation and lets the model do what it is better at, choosing the method and structuring the argument. Proofs remain the frontier, since a single invalid step quietly invalidates the whole chain.