maximum-entropy inverse RL

Maximum-entropy IRL gives a principled answer to the ambiguity problem: among all reward functions consistent with the demonstrations, prefer the one that explains them while assuming the expert is as random as the data allows. In plain terms, it models the expert as usually choosing high-reward paths but occasionally slipping, and refuses to read more structure into the demonstrations than is actually there.

Formally it places a probability on each trajectory proportional to the exponential of its total reward, so better paths are exponentially more likely but no path is impossible. Fitting the reward then becomes maximum-likelihood: adjust the reward weights so the demonstrated trajectories are as probable as possible under this model, which conveniently also matches the expert's feature expectations in expectation.

This framework resolves the degeneracy that plagued earlier IRL (it cannot collapse to the all-zero reward), handles noisy and suboptimal experts gracefully, and became the foundation for modern deep and adversarial IRL methods.