expert suboptimality

Expert suboptimality is the honest admission that real demonstrations are rarely perfect. The human teleoperating a robot is shaky, the gameplay logs mix brilliant moves with blunders, and different demonstrators disagree. Naive imitation faithfully copies all of it — including the mistakes — so a learner can inherit and even amplify the very flaws you hoped to avoid.

Approaches differ by how much side information you have. If demonstrations are ranked or scored, you can weight or rank them and learn to prefer the better behavior, sometimes even extrapolating beyond the best demonstrator. If you have a reward signal, demonstration-guided RL lets the agent treat demos as a hint rather than gospel and improve past them. Robust imitation methods instead down-weight outliers or model demonstrator noise explicitly.

The conceptual shift is to stop treating the expert as ground truth and start treating demonstrations as informative but fallible evidence about a goal you are still allowed to exceed.