Articulation Estimation

Infers an object’s parts, how they’re joined, which axis each part moves around, and how far it’s currently open.

Articulated objects — cabinet doors, drawers, laptops, scissors — are made of multiple rigid parts connected by joints. Articulation estimation infers, from an image or point cloud: how the object divides into parts, whether each joint is revolute or prismatic, the position and direction of each joint axis, and the current joint state (say, how many degrees a door is open). A robot needs this information before opening a door or pulling a drawer — pushing in the wrong direction can jam or damage the object. Notable work includes ANCSH, by Xiaolong Li, He Wang, Shuran Song, and colleagues, which estimates part poses, joint parameters, and joint states for unseen objects of a known category from a single depth point cloud; and Ditto (Zhenyu Jiang, Yuke Zhu, and colleagues, CVPR 2022), which uses two observations, before and after an interaction, to reconstruct part geometry and estimate a joint model whose result can be dropped straight into physics simulation — effectively building a digital twin of the articulated object.

ExampleFacing an unfamiliar cabinet, a robot first gives the door a push, compares the point clouds before and after, and estimates that the hinge is a vertical axis at the left edge of the cabinet — then pulls the door open along that arc.

Also called

Articulated Object Pose Estimation, Joint Parameter Estimation

Related

Articulated Object · Articulated Object Manipulation · PartNet-Mobility · Interactive Perception · Digital Twin · 6D Object Pose Estimation

Sources

Category-Level Articulated Object Pose Estimation (ANCSH, arXiv:1912.11913)
Ditto: Building Digital Twins of Articulated Objects from Interaction (arXiv:2202.08227)