goal-conditioned RL

Normally a policy learns one task with one fixed reward. A goal-conditioned policy instead learns a whole family of tasks at once by taking the goal as an explicit input: feed it 'go to the kitchen' and it goes to the kitchen, feed it 'go to the bedroom' and it goes there, all from the same network. The agent is no longer a specialist for one destination; it is a general controller you can point at any goal it has been trained to handle.

Concretely, both the policy and the value function gain an extra argument g, the desired goal, usually expressed as a target state or a point in some goal space. The reward is defined relative to the goal — typically a small bonus for reaching it and little else — so the agent learns 'how to reach things' in general rather than 'how to reach one thing'. This is exactly the interface a worker needs in a manager–worker hierarchy, which is why goal conditioning sits at the heart of hierarchical RL.

The big advantage is generalisation and reuse: skills learned for one goal transfer to nearby goals, and a single agent covers a continuum of tasks. The big difficulty is the sparse reward — most goals are reached rarely at first, so the agent sees almost no successful examples. That problem is precisely what hindsight relabeling and universal value functions are designed to solve.