HumanoidBench

A simulated humanoid benchmark from Berkeley and others, covering 27 whole-body locomotion and manipulation tasks.

HumanoidBench was proposed by researchers at UC Berkeley and Yonsei University (Carmelo Sferrazza, Pieter Abbeel, and colleagues), published at RSS 2024. Humanoid robot hardware is expensive and fragile, which makes algorithm research hard to run at scale. The benchmark builds a Unitree H1 fitted with two Shadow dexterous hands in MuJoCo, with 27 whole-body control tasks: 12 locomotion tasks (walk, run, hurdle, crawl, climb stairs, and more) and 15 manipulation tasks (open a cabinet, open a door, push, shoot a basketball, insert, and more). The codebase also provides variants such as an H1 without hands and a Unitree G1, along with DreamerV3, TD-MPC2, SAC, and PPO baselines. The paper found that the strongest reinforcement-learning algorithms at the time performed poorly on most tasks, while methods that first learn low-level skills like walking and reaching, then add hierarchical control on top, performed better.

ExampleThe same two-handed H1 has to complete locomotion tasks such as walk, hurdle, and stair, as well as manipulation tasks such as cabinet, door, basketball, and insert, all trained and scored with the same algorithm.

Also called

Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation, humanoid-bench

Related

Humanoid Robot · Unitree H1 · Shadow Dexterous Hand · MuJoCo (Multi-Joint dynamics with Contact) · Whole-Body Control · TD-MPC2

Sources

HumanoidBench (arXiv 2403.10506)
HumanoidBench project page
Robotics: Science and Systems XX (2024) proceedings

As of

2024-07