demonstration-guided RL

Demonstration-guided RL takes the best of both worlds: use a handful of demonstrations to get started fast, then let the agent improve past them with real reward. Pure imitation can never beat the demonstrator and pure RL can flail for ages before it stumbles onto reward in a hard task; combining them gives a warm start that exploration alone would take forever to find.

Common recipes seed the replay buffer with expert transitions, add an imitation loss that keeps the policy near the demonstrations early on and fades it out as the agent gets good, or use the demonstrations to shape rewards or bias exploration toward promising regions. The agent first learns to roughly mimic, then RL refines and ultimately surpasses the demonstrator on the true objective.

This is the practical sweet spot for sparse-reward and safety-sensitive problems — robotics, game-playing, dialogue fine-tuning — where demonstrations are affordable but not perfect, and the reward, though rare, is what you actually care about.