hindsight experience replay (HER)

Hindsight experience replay turns failures into lessons by exploiting a simple truth: even a botched attempt achieved some outcome. If you tried to reach goal A but ended at B, you can pretend in hindsight that B was the goal all along, and suddenly the same trajectory is a success you can learn from, even though the real reward was zero.

For goal-conditioned policies with sparse reward, HER stores each transition as usual and then adds relabeled copies whose goal is a state actually achieved later in the same episode, often the final state or a randomly chosen future one. The reward is recomputed under the substituted goal, so the replay buffer fills with achievable successes. Because relabeling does not depend on which policy generated the data, it works with any off-policy algorithm such as DQN, DDPG, or soft actor-critic.

In effect HER manufactures a dense, implicit curriculum out of sparse-reward tasks, without any hand-designed reward shaping.