Curriculum Learning

A training strategy that starts a model on easy samples or tasks and gradually increases the difficulty.

Curriculum learning was formally proposed by Bengio and colleagues in a 2009 ICML paper: like a student following a course syllabus, the model first sees easy samples or sub-tasks, and the difficulty is gradually increased. Their experiments showed this speeds up convergence and improves generalization. It's used heavily in robot reinforcement learning, because starting directly on the hardest version of a task means the policy almost never earns reward and can't learn at all. Common forms include terrain curricula (a legged robot practices on flat ground before stairs and slopes), gradually raising the commanded speed, and gradually widening the range of domain randomization; difficulty can also be adjusted automatically based on the policy's current performance, called an automatic curriculum.

Examplelegged_gym (Rudin et al., 2021) simulates 4,096 ANYmal quadrupeds at once: a robot that walks past the edge of its current terrain gets a harder terrain next episode, and one that covers less than half the target distance gets an easier one; step height ranges from 5 to 20 cm and slope from 0° to 25° over the curriculum.

Also called

Curriculum, Automatic Curriculum Learning

Related

Terrain Curriculum · Reinforcement Learning · Automatic Domain Randomization · Sparse Reward · legged_gym · Massively Parallel Reinforcement Learning

Sources

Curriculum Learning (Bengio et al., ICML 2009)
Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning (arXiv 2109.11978)