Cross-Embodiment Data

Data collected from many different kinds of robots and pooled together for training.

Cross-embodiment data refers to a collection of data drawn from multiple robot embodiments (a robot's specific physical hardware form) whose shape, degrees of freedom, camera placement, and action space all differ — single arms, dual arms, mobile bases, and humanoids among them. A single lab's data is limited in volume, so pooling data from many labs is meant to help a model learn skills that don't depend on any one particular robot, benefiting every embodiment involved. The leading example is Open X-Embodiment, led by Google DeepMind in 2023: 60 datasets from 34 labs, 22 kinds of robots, and more than 1 million trajectories, unified into RLDS format. The difficulty is that different labs' actions and observations don't line up, requiring normalization, a unified action space, or an embodiment-specific output head for each robot; a poorly chosen mixture can even cause negative transfer.

ExampleRT-1-X, trained on Open X-Embodiment, averaged about a 50% higher success rate than the original methods each lab trained using only its own data, on several tasks from labs with small datasets.

Also called

Multi-Embodiment Data, X-Embodiment Data

Related

Cross-Embodiment · Open X-Embodiment · Embodiment Gap · Unified Action Space · Embodiment-specific Head · Positive / Negative Transfer

Sources

Open X-Embodiment Collaboration 2023: Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Open X-Embodiment 项目页 (Chinese)