feature expectation matching

Feature expectation matching is the engine inside feature-based inverse RL. Pick a vector of features describing each state, then summarize any policy by its feature expectations — the discounted total of those features the policy accumulates on average as it acts. The expert's demonstrations give you a target vector; the goal is to find a policy whose own feature-expectation vector lands on top of it.

Why this works is a neat piece of algebra: if a reward is linear in the features, then a policy's return is the dot product of the reward weights with its feature expectations. Match the feature expectations and you match the return under every linear reward — so you behave as well as the expert without ever needing to know which reward they were really optimizing.

In practice you minimize the distance between the two feature-expectation vectors, often by repeatedly solving an RL problem under a current reward guess and nudging the weights toward whichever features the expert used more. It is the concrete optimization target shared by apprenticeship learning and max-entropy IRL.