reality gap

The reality gap is the discrepancy between how a system behaves in simulation and how it behaves in reality. A walking policy that strides confidently across the simulated floor may stumble on the real robot because the simulated joints, friction, and timing do not quite match the physical ones.

The gap comes from unmodeled or mis-estimated dynamics (contact, friction, motor backlash), sensor and actuation noise, communication latency, and visual differences between rendered and real images. Because RL exploits every quirk of its training environment to maximize reward, it eagerly latches onto sim-only artifacts that simply do not exist in the real world.

There is no single cure. The gap is fought from both sides at once: making the simulator more faithful through system identification, and making the policy less sensitive to simulator details through domain randomization and robustness training.