Continual Learning

Letting a model learn a sequence of new tasks and new data without forgetting what it already learned.

Continual learning studies how a model can keep accumulating and updating knowledge as it continually receives new tasks and new data throughout its whole deployment lifetime. The biggest obstacle is catastrophic forgetting: a neural network's performance on old tasks often drops sharply once it keeps training on new data. The core trade-off is the stability-plasticity balance — protecting old knowledge while still being able to learn new things. Common approaches include regularization (such as DeepMind's 2017 EWC, which slows updates to weights important for old tasks), replay (mixing old data, or generated stand-ins for it, back into training), and structural expansion (adding separate parameters for each new task). For robots, deployment means constantly running into new objects and new scenes, and the ideal is learning on the fly rather than retraining from scratch every time; the 2023 LIBERO benchmark was designed specifically for lifelong robot learning, with 4 task suites totaling 130 tasks. A VLA losing its original VLM's language ability and common sense after fine-tuning is also a form of forgetting.

ExampleA home robot first learns to fold towels, then later learns to open the refrigerator and wash dishes; continual learning requires that after it learns dishwashing, its success rate at folding towels doesn't drop noticeably.

Also called

Lifelong Learning, Incremental Learning

Related

Catastrophic Forgetting · Transfer Learning · Fine-tuning · Multi-Task Learning · LIBERO Benchmark · Knowledge Insulation

Sources

A Comprehensive Survey of Continual Learning: Theory, Method and Application (arXiv 2302.00487)
Overcoming catastrophic forgetting in neural networks (EWC, arXiv 1612.00796)
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning (arXiv 2306.03310)