inverse reinforcement learning (IRL)

Inverse reinforcement learning flips the usual setup. Instead of being given a reward and finding a policy, you watch an expert behave and try to recover the reward function that would make their behavior optimal. It is the natural tool when a goal is hard to write down but easy to demonstrate, such as smooth, courteous driving.

The problem is fundamentally ill-posed, because many reward functions explain any given behavior, including the trivial all-zero reward. Classic resolutions add structure: max-margin methods demand a reward under which the expert beats alternatives by a margin, feature-expectation matching requires the learner's discounted features to match the expert's, and maximum-entropy IRL selects the least committal reward consistent with that matching, resolving the degeneracy with an entropy prior. Adversarial imitation methods such as GAIL and AIRL sidestep explicit recovery, matching occupancy distributions with a discriminator.

Even at best the reward is identifiable only up to ambiguities like potential-based shaping and scaling, which is why an explicit prior or regularizer is essential.