DexWild

CMU's portable hand-data-collection system that lets ordinary people gather dexterous-manipulation data with their own hands in real-world settings.

DexWild is a dexterous-manipulation data-collection and training approach from Deepak Pathak's group at Carnegie Mellon University, published at RSS 2025. Teleoperated data is high quality but expensive, and it's hard to cover enough different environments this way; DexWild instead has ordinary people wear a low-cost, portable device called the DexWild-System and collect data with their own hands in real-world settings. A motion-capture glove measures finger pose; markers on the glove are located by a tracking camera to capture wrist position; two cameras mounted on the palm capture a close-up view in which the hand itself is barely visible, which makes the footage easier to reuse across different robot embodiments. Ten untrained collectors gathered 9,290 demonstrations across 93 environments, at roughly 4.6 times the speed of teleoperation. After co-training on this human hand data together with a smaller amount of robot data, the resulting policy reaches 68.5% success in unseen environments — about 4 times the success rate of training on robot data alone.

ExampleA clothes-folding task was co-trained on about 1,124 human hand demonstrations plus 290 robot demonstrations; the training used a LEAP hand mounted on an xArm, and the resulting policy also transferred zero-shot to a Franka arm.

Also called

DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies, DexWild-System

Related

In-the-wild Data · Co-training · Data Glove · Cross-Embodiment · LEAP Hand · Robot-free (Embodiment-free) Data Collection

Sources

DexWild 项目主页 (Chinese)
DexWild (arXiv)

As of

2025-05