Sim-to-Real Gap (Reality Gap)

The mismatch between simulation and reality that makes a policy trained in sim perform worse on a real robot.

The sim-to-real gap refers to the mismatch between a simulated environment and the real world, which causes a policy trained in simulation to perform worse, or fail outright, once it's transferred to a real robot. The gap has several sources: inaccurate physical parameters (friction, mass, damping, motor characteristics), simplifications in the physics modeling itself (such as soft contact or flexible objects), visual differences (rendered lighting and textures don't match a real camera), and factors that simulation doesn't model at all, such as sensor noise and control latency. Because simulated training is cheap, safe, and can run at massive parallel scale, closing this gap is the central problem in sim-to-real transfer. Common countermeasures include system identification (calibrating simulation parameters to match reality), domain randomization (training the policy to handle a wide range of parameters), domain adaptation (shifting simulated data's distribution to look more like real data), and fine-tuning with a small amount of real-robot data.

ExampleA quadruped locomotion policy trained in simulation shakes in place or even falls over on the real robot because the real motors respond more slowly than the simulated ones did — a textbook case of the sim-to-real gap.

Also called

Reality Gap, Sim2Real Gap

Related

Sim-to-Real Transfer · Domain Randomization · System Identification · Domain Adaptation · Actuator Modeling (Actuator Network) · Real-to-Sim

Sources

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (arXiv 2009.13303)
Lilian Weng: Domain Randomization for Sim2Real Transfer