In-the-wild Data

Data collected in homes, offices, outdoors, and other uncontrolled real-world settings, rather than a fixed lab bench.

In-the-wild data borrows a term from computer vision, meaning data collected in real settings outside the lab, where lighting, background, and object placement are all uncontrolled and every scene differs from the last. Most robot data used to come from a handful of fixed lab tabletops, so models easily failed in a new room; researchers have since moved collection into real environments to improve scene generalization. The difficulty is that a real robot isn't easy to move around, so portable alternatives have emerged: DROID mounts a Franka arm on a movable height-adjustable cart and collected 76,000 trajectories across 564 scenes; UMI uses a handheld gripper, and DexWild uses a bare human hand plus a palm-mounted camera — neither needs a robot physically on-site.

ExampleCarnegie Mellon's DexWild collected 9,290 human demonstrations across 93 different environments; co-trained with robot data, the resulting policy's success rate in new environments was about 4 times that of a policy trained on robot data alone.

Also called

Real-world Scene Data

Related

DROID (Distributed Robot Interaction Dataset) · Universal Manipulation Interface · DexWild · Scene Generalization · Data Diversity · Robot-free (Embodiment-free) Data Collection

Sources

DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset
DexWild 项目主页 (Chinese)
Universal Manipulation Interface (UMI) 项目主页 (Chinese)