Whole-Body Teleoperation

One operator simultaneously controls a robot's arms, torso, and legs or base — its whole body at once.

Ordinary teleoperation usually only controls the arm; whole-body teleoperation lets one operator simultaneously direct the robot's upper body, torso, and legs or mobile base, used to collect data for tasks that need the whole body to work together, such as reaching for something while walking or crouching to pick something up. There are two common forms: a mobile manipulation platform, such as Stanford's 2024 Mobile ALOHA, where the operator is physically tethered to the base, controlling both arms with a leader-follower setup and steering the base with their own body; and a humanoid robot, where mocap gear, a VR headset, or an exoskeleton captures the operator's whole-body motion, which is mapped through motion retargeting onto the robot and tracked by a reinforcement-learning-trained whole-body controller that keeps balance, as in OmniH2O (2024) and Stanford's TWIST (2025). The difficulty is that body proportions and balance capability differ between human and robot, and the operator usually has no force feedback either.

ExampleIn OmniH2O, an operator can wear just a single VR headset, driving a full-size humanoid's whole-body motion with head and hand pose alone; the recorded data forms the OmniH2O-6 dataset.

Also called

Humanoid Whole-body Teleop

Related

Teleoperation · Motion Retargeting · Whole-Body Control · OmniH2O · Twist · Mobile ALOHA

Sources

Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
TWIST: Teleoperated Whole-Body Imitation System

As of

2025-05