exploration in sparse-reward settings

In many real tasks reward is almost silent: you get nothing for thousands of steps and then a single point at the very end, if you ever stumble onto success at all. With no reward gradient to follow, a value-maximising agent has nothing to climb — random flailing essentially never trips the reward, so it never starts learning. This is where naive exploration like epsilon-greedy quietly collapses.

The standard rescues all manufacture a denser signal. Intrinsic motivation — curiosity, novelty, or count bonuses — gives the agent something to chase before the real reward ever appears. Reward shaping hand-crafts intermediate guidance. Other approaches change the data the agent learns from: hindsight relabelling pretends whatever it reached was the goal, while curricula and demonstrations seed it near success. The unifying idea is to make progress measurable when the environment itself refuses to say so.