meta-reinforcement learning

Most RL agents start every new task as a blank slate. Meta-reinforcement learning asks a sharper question: can an agent learn how to learn, so that when it meets a new task drawn from a familiar family, it adapts in a handful of episodes instead of millions of steps? The image is a lab mouse that has run many mazes before — it does not solve a new maze from scratch, it knows how mazes tend to work and probes efficiently.

Formally we train across a distribution of tasks (different reward functions or dynamics) so the agent acquires an adaptation procedure, not just one policy. Two big families exist. Gradient-based methods, such as learning a good initialization, tune the network so a few gradient steps specialize it. Context-based or recurrent methods feed recent experience into a memory — an RNN or transformer — that implicitly infers which task it is in and shifts behavior accordingly, sometimes described as learning a reinforcement learning algorithm inside the weights.

Meta-RL is exciting because fast adaptation is exactly what real deployment needs, but it inherits hard problems: it works only when test tasks really come from the training distribution, the meta-training itself is data-hungry and unstable, and rewarding exploration during adaptation (the agent must sometimes act to gather information, not to score) stays delicate. It overlaps heavily with the broader idea of meta-learning.