Positive / Negative Transfer

When adding other tasks or data makes the target task better, that's positive transfer; when it makes it worse, that's negative transfer.

This is a pair of concepts from transfer learning and multi-task learning: applying knowledge learned on a source task or source data to a target task, where the target task's performance improving is positive transfer, and it getting worse is negative transfer. Wang and colleagues gave a formal definition of negative transfer in a CVPR 2019 paper, noting it tends to happen when the source data is too weakly related to the target task. In embodied AI, whenever data from multiple robots, multiple tasks, or internet image-text sources is trained together, the question of whether the added data helps or hurts always comes up. Common causes of negative transfer include unrelated tasks, mismatched action spaces or coordinate frames across different robot embodiments, a poorly balanced data mix, and insufficient model capacity. Responses include tuning the data mixing ratio, giving each embodiment its own output head, and freezing or isolating parts of the model.

ExampleIn the Open X-Embodiment experiments, RT-1-X, trained on data from 22 robot types, beat single-robot training by 50% on average on robots with little of their own data — positive transfer. But on robots with plenty of data, RT-1-X underfit and did worse than RT-1 trained only on that robot's own data; only the much larger RT-2-X regained the advantage.

Also called

Positive Transfer, Negative Transfer

Related

Transfer Learning · Multi-Task Learning · Cross-Embodiment · Co-training · Data Mixture · RT-X

Sources

Characterizing and Avoiding Negative Transfer (CVPR 2019)
Open X-Embodiment: Robotic Learning Datasets and RT-X Models (project page)
Open X-Embodiment paper (arXiv HTML)