generative adversarial imitation learning (GAIL)

GAIL borrows the trick behind generative adversarial networks and points it at imitation. Two networks play a game: a discriminator tries to tell apart state-action pairs that came from the expert from those produced by the learner, while the policy tries to fool it by acting so convincingly that the discriminator cannot tell who is who. When the policy wins, its behavior is statistically indistinguishable from the expert's.

The clever part is what the policy optimizes. The discriminator's verdict — how expert-like a given action looks — is fed back as a reward, and the policy is trained with ordinary RL to maximize it. So GAIL never explicitly recovers a reward function or copies actions point by point; it directly matches the learner's whole state-action distribution to the expert's, which is exactly what defeats covariate shift.

The result is sample-efficient in expert demonstrations and robust on long horizons, at the cost of adversarial training's usual instability and the need for environment interaction during learning.