continual reinforcement learning

Imagine teaching an agent task A, then task B, then task C, one after another, with no chance to revisit A. Continual reinforcement learning studies exactly this stream-of-tasks setting, and its defining villain is catastrophic forgetting: neural networks tend to overwrite the weights that encoded task A while learning task B, so the agent quietly loses skills it once had. The goal is to keep learning new things without erasing the old.

This is harder in RL than in supervised learning because the data is non-stationary by nature — the agent's own changing policy shifts what it sees — so the line between normal learning and harmful forgetting is blurry. The standard toolkit borrows from continual learning broadly: replay buffers that rehearse old experience, regularizers that protect important weights, and modular or expanding architectures that give new tasks their own capacity. Each trades off stability (remembering) against plasticity (adapting), the central tension of the field.

It matters because any agent meant to live in the real world will face tasks in sequence, not all at once, and cannot store every experience forever. Continual RL is closely tied to lifelong RL; the line between them is soft, but continual RL tends to emphasize the forgetting problem over a known sequence, while lifelong RL emphasizes open-ended accumulation and reuse of skills.