delayed reward

In many problems the payoff for a good action shows up much later, not immediately. A clever opening move in chess might only matter twenty moves on; studying tonight pays off at next month's exam. The reward right after an action often says little about whether the action was wise—its real consequences are still downstream.

Delayed reward is a defining difficulty of RL and the reason it needs special tools. The agent must connect a reward back to the earlier decisions that earned it, and it must value future reward, not just instant gratification. The discount factor and value functions exist largely to handle this: they let the agent estimate the long-term worth of an action even when the actual reward is far away.