Symmetry Augmentation

Using a robot's left-right symmetry, by mirroring data or adding a loss term, to make its learned motion symmetric.

Most legged and humanoid robots are left-right symmetric, and an ideal policy should satisfy: mirror the state left-to-right, and the output should be exactly the mirror of the original action. Reinforcement learning from scratch, however, often learns an asymmetric gait, for instance dragging one leg. Two fixes are common. The mirror loss, proposed by Yu, Turk, and Liu at SIGGRAPH 2018, adds a term to the loss function penalizing the difference between the policy's output on a mirrored state and the mirror of its original output. Symmetry data augmentation instead mirrors every collected sample and trains on both copies. ETH's Mittal and colleagues compared the two at ICRA 2024 and found that data augmentation converges faster and reaches higher return within PPO. The rsl_rl library now has both options built in.

ExampleMittal and colleagues used symmetry data augmentation to train an ANYmal quadruped to climb boxes; it reached higher return than a plain PPO baseline with no symmetry handling, varied less across random seeds, and was deployed on the real robot.

Also called

Mirror Loss, Mirror-Symmetry Augmentation, Symmetry Loss

Related

Data Augmentation · RL-based Locomotion Control · Proximal Policy Optimization · rsl_rl · Legged Locomotion · Loss Function

Sources

Yu, Turk, Liu 2018: Learning Symmetric and Low-energy Locomotion (SIGGRAPH 2018)
Mittal et al. 2024: Symmetry Considerations for Learning Task Symmetric Robot Policies (ICRA 2024)
GitHub: leggedrobotics/rsl_rl ppo.py（symmetry_cfg / use_mirror_loss）