DexterityGen

Meta and Berkeley's low-level dexterous-hand controller that turns a human's rough teleoperation intent into precise finger motion.

DexterityGen (DexGen for short) is a dexterous-hand control method released in February 2025 by teams at Meta FAIR and UC Berkeley. A dexterous hand has many degrees of freedom, and direct human teleoperation struggles to reliably perform fine motions like spinning a pen in-hand or turning a screw; pure reinforcement learning, on the other hand, has trouble learning long, complex tasks. Its division of labor: first, reinforcement learning in simulation trains a huge library of in-hand motion primitives — rotation, translation, and so on — collecting about 10^10 state transitions; that data then trains a diffusion model as a “foundation controller,” which generates fingertip keypoint motion from the current observation, converted into joint commands by an inverse-dynamics model. At deployment, a higher-level source (such as human teleoperation) only needs to give a rough intent, and DexGen refines it into stable, dexterous motion, while also being able to reject dangerous actions.

ExampleOn a Franka arm fitted with an Allegro dexterous hand, an operator wearing a Manus data glove teleoperates with DexGen's help to pick up and use a pen, a syringe, and a screwdriver; the paper reports the object stays gripped without dropping 10 to 100 times longer than baselines.

Also called

DexGen

Related

Dexterous Manipulation · In-hand Manipulation · Reinforcement Learning · Diffusion Model · Teleoperation · Meta Fundamental AI Research

Sources

DexterityGen: Foundation Controller for Unprecedented Dexterity (arXiv:2502.04307)
DexGen 项目主页 (Chinese)

As of

2025-02