skill discovery

Imagine dropping an agent into a world with no task and no reward and telling it: just go learn to do interesting, distinct things. Skill discovery is exactly this — learning a repertoire of reusable behaviours before any specific task is set, purely from interaction. The result is a set of skills, indexed by some latent code, that the agent can later call on when a real objective finally arrives, much like a child mastering walking, grabbing and climbing long before being asked to fetch a toy.

Without an external reward you need an internal one, so these methods invent intrinsic objectives. The most influential family rewards skills for being different from one another and predictable in their effects: each skill should drive the agent to a recognisably distinct region of behaviour, so that you could guess which skill is running just by watching. Maximising that distinguishability, often phrased as a mutual-information objective between skills and the states they visit, yields a diverse, controllable set of behaviours with no task reward at all.

The promised payoff is huge: a library of skills pre-trained once and reused across many downstream tasks, shrinking the learning each new task needs. The honest caveats are that unsupervised skills often cover the easy, high-coverage parts of a space rather than the task-relevant ones, and that bridging from self-generated skills to a specific goal can still require substantial fine-tuning.