Articulated Object Manipulation

Manipulating objects with movable joints, such as opening a cabinet door, pulling a drawer, or lifting a laptop lid.

An articulated object is made of multiple parts connected by rotating or sliding joints — a cabinet door rotating on a hinge, a drawer sliding on rails, a laptop, a faucet, a refrigerator door. To manipulate one, a robot must not only find the handle but also infer the joint type, the location of the axis, and the direction of motion, then apply force along the constrained direction as it moves — otherwise it will get stuck or even break the handle off. This is a fundamental capability for household robots. Research commonly uses the SAPIEN simulator together with the PartNet-Mobility dataset of articulated objects; a notable example is Where2Act (ICCV 2021), which learned to predict where on an object it could push or pull through repeated interaction in simulation.

ExampleOpening a microwave the robot has never seen before: it first recognizes the door handle, infers that the door rotates around a vertical axis on the left, then pulls the door open along the arc centered on that axis.

Related

Articulated Object · Articulation Estimation · PartNet-Mobility · SAPIEN (SimulAted Part-based Interactive ENvironment) · Affordance · Manipulation

Sources

Where2Act: From Pixels to Actions for Articulated 3D Objects
SAPIEN: A SimulAted Part-based Interactive ENvironment