OakInk

A Shanghai Jiao Tong University hand-object interaction dataset, annotated with object use and two-handed manipulation.

OakInk is a series of hand-object interaction datasets from Cewu Lu's group at Shanghai Jiao Tong University. In the first version (CVPR 2022), the “Oak” part annotates affordances (how an object can be used) for 1,800 everyday objects, while the “Ink” part records real human grasps on 100 of those objects and transfers them to virtual objects, yielding about 50,000 interactions labeled with intended use. The second version, OakInk2 (CVPR 2024), shifts to two-handed completion of complex tasks: 627 sequences and 4.01 million multi-view image frames, annotating the 3D pose of the body, both hands, and the objects, and organizing tasks into three layers — affordance, atomic task, and complex task. It's commonly used for hand pose estimation, grasp generation, and synthesizing two-handed motion.

ExampleOne OakInk2 sequence breaks a complex task down into atomic subtasks like picking up, pouring, and setting down; every frame is labeled with the 3D pose of both hands (as MANO parameters) and the object.

Also called

OakInk2, OakInk Dataset

Related

Hand-Object Interaction · MANO · Affordance · Bimanual Manipulation · DexYCB · ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

Sources

OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction (arXiv 2203.15709)
OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion (arXiv 2403.19417)
OakInk2 项目主页 (Chinese)

As of

2024-12