Terrain Curriculum

In legged reinforcement learning, a training schedule that gradually moves a robot onto harder terrain as it improves.

Terrain curriculum is curriculum learning applied specifically to legged locomotion. Training from scratch directly on steep slopes or stairs makes the robot fall almost constantly, yielding little useful learning signal; training only on flat ground never teaches it complex terrain either. ETH's Rudin and colleagues proposed a “game-inspired” curriculum in 2021's legged_gym: terrain is arranged in a grid of rising difficulty, and after each episode, if the robot walked past half the length of its tile it moves up a level, if it didn't even cover half the commanded distance it moves down a level, and reaching the hardest level sends it back to a random earlier one to avoid forgetting easy terrain. Thousands of parallel environments each move up and down independently, so overall difficulty automatically tracks the policy's skill. Isaac Lab's velocity-tracking task follows the same rule.

Examplelegged_gym by default splits terrain into 10 difficulty levels across 20 columns, with types including smooth slopes, rough slopes, stairs up, stairs down, and discrete obstacles; a robot starts at a random level between 0 and 5, then moves up or down based on how far it walks each episode.

Also called

Game-inspired Curriculum

Related

Curriculum Learning · RL-based Locomotion Control · legged_gym · Massively Parallel Reinforcement Learning · Rough-terrain Locomotion · Heightfield Terrain

Sources

Rudin et al. 2021: Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning (CoRL 2021)
GitHub: leggedrobotics/legged_gym legged_robot.py（_update_terrain_curriculum）
GitHub: isaac-sim/IsaacLab locomotion velocity curriculums.py（terrain_levels_vel）