open-ended learning

Most learning has a finish line — a fixed task, a target score, a point where you are done. Open-ended learning refuses the finish line: it studies systems that endlessly invent new challenges and master them, generating their own ever-harder curriculum so that learning never plateaus. The inspiration is evolution and human culture, which have produced unbounded novelty for billions of years with no external goal handed in from outside.

The usual machinery is a co-evolving loop between agents and the problems they face: an environment generator proposes new tasks or levels, agents learn to solve them, and the generator is pushed toward challenges that are neither too easy nor impossible — at the frontier of current ability. Methods like POET and procedurally expanding game worlds embody this, often paired with population-based training so a diversity of agents and niches keeps the system from collapsing onto one trick.

Open-ended learning is one of the most ambitious bets in RL: if it worked at scale, an agent could keep getting more capable indefinitely with little human curation. But it is also among the least settled — measuring genuine open-ended progress is itself unsolved, runs can stagnate or drift into uninteresting novelty, and we lack theory for what makes a generator reliably productive. It ties closely to lifelong learning and curriculum design.