Sim-to-Sim Transfer

Running a policy trained in one simulator inside a different simulator, to expose problems before they reach the real robot.

Sim2Sim is common in reinforcement-learning pipelines for legged and humanoid robots: a policy is trained at scale in a GPU-parallel simulator such as Isaac Gym or Isaac Lab, then dropped unchanged into a different simulator, such as MuJoCo, which has a different contact model and solver. If the policy becomes unstable once the simulator changes, that means it was exploiting quirks specific to the original simulator, and deploying it straight to the real robot would be risky. Unitree's unitree_rl_gym documents its pipeline as “train → replay → Sim2Sim → Sim2Real”; RobotEra's Humanoid-Gym, open-sourced in 2024, likewise provides an Isaac Gym to MuJoCo validation pipeline. It is cheap and carries no risk of damaging a real robot, but passing it still does not guarantee success on the real robot.

ExampleAfter training a walking policy for the Unitree G1 in Isaac Gym with unitree_rl_gym, developers first run its Sim2Sim script to check the policy in MuJoCo before deploying it to the real robot.

Also called

Sim2Sim, Cross-Simulator Validation

Related

Sim-to-Real Transfer · Sim-to-Real Gap (Reality Gap) · Isaac Gym · MuJoCo (Multi-Joint dynamics with Contact) · Humanoid-Gym · unitree_rl_gym (Unitree RL Gym)

Sources

unitreerobotics/unitree_rl_gym GitHub
roboterax/humanoid-gym GitHub
Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer (arXiv 2404.05695)