unsupervised RL

Normally RL needs a reward to learn — but what if no reward has been given yet? Unsupervised RL is the idea of letting an agent practice in an environment before any task is defined, so that when a reward finally arrives, it already knows how to move, explore, and reach states efficiently. The analogy is a child at play: nobody hands a toddler a scoreboard, yet hours of aimless play build motor skills and a model of how the world responds.

Without an external reward, the agent invents its own signal. Common objectives are intrinsic motivation that rewards novelty or surprise, empowerment that rewards reaching many distinct future states, and skill-discovery methods that learn a diverse set of distinguishable behaviors. The output is not a finished policy but a pretrained set of skills, a good state representation, or an explored replay buffer — a foundation that a short reward-driven fine-tuning phase can then specialize.

This reframing — unsupervised pretraining then quick task adaptation — is the same recipe that transformed vision and language, and it directly attacks the sample-efficiency and exploration problems. The open questions are which intrinsic objective produces the most reusable behaviors, and how to guarantee that pretraining actually helps the downstream task rather than wandering into useless corners of the environment.