process vs outcome rewards

When a model solves a multi-step problem, you can reward it in two different places. An outcome reward looks only at the final answer: right or wrong, full stop. A process reward looks at each step of the reasoning and scores whether each one is sound. The difference matters most for tasks like math and code, where you can reach a correct answer through a flawed or lucky route.

Outcome rewards are cheap and objective but coarse: they happily reinforce a wrong line of reasoning that stumbles into the right answer, and give no signal about where a failed attempt went off track. Process rewards, supplied by a process reward model that grades steps, give denser feedback and discourage plausible-looking but invalid reasoning — but they are far more expensive to label and can be gamed into reasoning that merely looks rigorous.

This trade-off sits at the heart of training reasoning models. Process supervision tends to produce more trustworthy, checkable reasoning, while outcome supervision is simpler to scale, and many systems blend the two.