RH20T

A multimodal real-robot manipulation dataset from Shanghai Jiao Tong University, with force, audio, and human demonstration video.

RH20T is a real-robot manipulation dataset released in 2023 by Cewu Lu's group (the MVIG lab) at Shanghai Jiao Tong University. It contains over 110,000 contact-rich manipulation sequences covering 147 tasks (48 drawn from RLBench, 29 from Meta-World, and 70 designed by the authors), collected across 7 different robot configurations. Beyond multi-view RGB and depth images, each sequence also includes 6-axis force/torque readings, audio, and action records, with fingertip tactile data on some configurations; every robot sequence is also paired with a video of a human performing the same task plus a language description, aimed at supporting one-shot imitation learning — learning a task from watching a single human demonstration. The full dataset totals about 40TB.

ExampleOne RH20T sequence provides synchronized RGB-D footage from multiple cameras, 100Hz force/torque readings, dual-microphone audio, and a video of a human performing the same task.

Also called

RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot

Related

SJTU MVIG Lab · Multimodal Data · Contact-rich Manipulation · Six-Axis Force/Torque Sensor · Real-Robot Data · One-shot Imitation Learning

Sources

RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot (arXiv:2307.00595)
RH20T 项目主页 (Chinese)

As of

2023-09