Gesture Recognition

Lets a machine recognize a gesture a person makes from images or sensor signals and understand its meaning.

Gesture recognition is a task in computer vision and human-robot interaction: recognizing a person’s hand gesture — a fist, a thumbs-up, pointing somewhere — from an ordinary camera, a depth camera, a data glove, or EMG signals. A static gesture looks only at the hand’s shape at one moment; a dynamic gesture needs to look at a motion trajectory over time, like waving. The common approach today is to detect hand keypoints first, then classify based on those keypoints. It differs from hand pose estimation: the latter recovers the complete 3D hand shape, while gesture recognition only outputs a category. In embodied AI, gesture recognition is used for human-robot interaction — for example, using a gesture to make a robot stop, follow, or fetch a pointed-at object; teleoperation systems also map specific gestures to control commands, such as opening and closing a gripper or switching modes.

ExampleGoogle’s MediaPipe Gesture Recognizer can recognize a closed fist, an open palm, a pointing index finger, thumbs up, thumbs down, a victory sign, and more by default, while also outputting hand keypoints, which can be used to implement “open palm makes the robot stop.”

Also called

Hand Gesture Recognition

Related

Hand Pose Estimation · Human-Robot Interaction · Keypoint Detection · MediaPipe · Data Glove · Teleoperation

Sources

Wikipedia: Gesture recognition
MediaPipe Gesture Recognizer 官方文档 (Chinese)