inverse reinforcement learning

Inverse reinforcement learning flips ordinary RL on its head. Normal RL takes a reward function and searches for the best behavior; IRL takes the behavior — expert demonstrations — and searches for the reward function that would make that behavior optimal. Instead of copying what the expert does, it tries to recover why they do it: the hidden objective driving their choices.

The payoff is that a reward is far more portable than a cloned policy. Once you have inferred the expert's goal, you can run standard RL against it to act well even in states or dynamics the demonstrations never showed, transfer the goal to a new robot body, or combine it with planning. The classic formulation looks for a reward under which the expert's expected return is at least as high as any alternative policy's.

The deep difficulty is ill-posedness: countless reward functions explain the same demonstrations, including the trivial all-zero reward that makes every policy optimal. Practical IRL therefore needs extra assumptions — feature matching, a maximum-entropy prior, or an adversarial game — to single out a sensible reward.