Interactive Perception

A robot actively pushes, pokes, or pulls objects, using the resulting changes to understand its environment.

Interactive perception means a robot changes the scene through physical contact — pushing, poking, pulling, picking something up to look at it — and uses the sensory changes caused by that action to understand the environment. A 2017 survey by Bohg, Hausman, Brock, Kragic, and colleagues in IEEE T-RO systematically laid out this direction, with two core ideas: interaction produces signals that passive observation can’t get, and knowing what action you just took lets you better predict and interpret the signal that follows. It differs from active perception, which usually just moves the camera or changes viewpoint without changing the environment; interactive perception actually changes the environment by acting on it. Typical uses include segmenting objects that are pressed together, estimating the joint axis of a drawer or cabinet door, and judging physical properties like an object’s mass and friction.

ExampleSeveral blocks sit pressed together, and looking at the image alone can’t tell their boundaries apart; the robot gives them a push, and the pixels that move together belong to the same block, letting it segment them. Tracking a handle’s motion while pulling open a drawer can likewise estimate the drawer’s sliding direction.

Related

Active Perception · Articulation Estimation · Affordance · Perception-Action Loop · Non-prehensile Manipulation · Embodied Perception

Sources

arXiv 1604.03670: Interactive Perception: Leveraging Action in Perception and Perception in Action（IEEE T-RO 2017）