Visual Servoing

Feeding image error straight into the control loop, adjusting the robot's motion in real time as the camera watches.

Visual servoing is a control method that uses visual information as feedback to drive robot motion in real time; related research dates back to around 1979, though the term ‘visual servo’ itself wasn't coined until 1987. The core idea is driving the error e = s − s* to zero, where s is the current feature extracted from the image (e.g., the pixel coordinates of a few corner points) and s* is the desired feature. The classic control law is v = −λL⁺e, where v is camera velocity, λ a gain, L the interaction matrix (describing how the features change as the camera moves), and L⁺ its pseudoinverse. It splits into two types by feature: image-based visual servoing (IBVS) works directly with pixel features and never needs to estimate a 3D pose; position-based visual servoing (PBVS) first estimates the target's 3D pose and then controls from that. The camera can be mounted on the robot's hand (eye-in-hand) or fixed externally (eye-to-hand).

ExampleA typical example from Chaumette and Hutchinson's 2006 tutorial: a wrist-mounted camera looks at four points arranged in a square, uses the difference between their current and desired pixel coordinates as the error, and computes camera velocity via v = −λL⁺e until the four points land at their desired image positions — at which point the camera has also reached the target pose.

Also called

Visual Servo Control, Vision-Based Robot Control

Related

Image-Based Visual Servoing · Position-Based Visual Servoing · Eye-in-Hand · Eye-to-Hand · Visuomotor Policy · Closed-loop Control

Sources

Chaumette & Hutchinson, Visual Servo Control Part I: Basic Approaches (IEEE RAM, 2006)
Wikipedia: Visual servoing