credit assignment problem

When a reward finally arrives—say, you win the game—which of the dozens of earlier moves deserves the credit? This is the credit assignment problem: spreading responsibility for an outcome back over the actions that led to it. It's the puzzle a coach faces reviewing a match, trying to tell the decisive play from the merely incidental one.

There are two faces to it. Temporal credit assignment asks which past time steps mattered, when reward is delayed across many steps. Structural credit assignment asks which part of the agent's policy to adjust. Much of RL—value functions, temporal-difference learning, policy gradients—is machinery for solving this well, because without it the agent can't turn a sparse, delayed reward into useful learning.