Occlusion

A target being partly or fully blocked from a sensor's view by something else, including the robot's own body.

Occlusion means part or all of a target is blocked by something else, so a sensor can't see it fully. Three scenarios are common in robotics: objects blocking each other (cluttered parts), the robot blocking its own view (the arm reaching in front of an object, or a dexterous hand's fingers blocking an object in the palm — called self-occlusion), and, during manipulation, the hand and the object blocking each other. Occlusion causes missing or wrong values in detection, segmentation, pose estimation, and depth, and a blocked object can effectively ‘disappear’ from a policy's input, leading to misjudgment. Because the location, size, and proportion of occlusion all vary unpredictably, it's one of the main reasons detection models still fall short of human performance. Common countermeasures include multiple camera views, a wrist camera, actively moving the viewpoint (active perception), adding touch or sound, or having the model complete or remember the blocked part.

ExampleAn arm relying only on a top-down camera to grasp a cup has its own arm block the cup right as the end-effector gets close, so the policy loses sight of the cup's position; adding a wrist camera fills in that missing view.

Also called

Self-Occlusion, Mutual Occlusion

Related

Multi-View · Wrist Camera · Active Perception · Point Cloud Completion / Shape Completion · Visuo-Tactile Fusion · Object Tracking

Sources

Occlusion Handling in Generic Object Detection: A Review (SAMI 2021)
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation（视觉易受遮挡） (Chinese)
Vision-Based Manipulators Need to Also See from Their Hands (ICLR 2022)