Success Detector

A model that judges whether a robot completed a task in a given episode, often used to give reward in RL.

A success detector takes an observation, usually a camera image, sometimes with the task instruction, and outputs success or failure, or a probability of success. Reinforcement learning, automatic data collection, and automatic evaluation all need to know whether a task got done, and the real world has no simulator-style ground truth for that, so people either watch it themselves or train a model to watch it. Two approaches are common: training a binary classifier for a single task, as HIL-SERL does, teleoperating roughly 200 successful and 1,000 failed examples per task; or directly querying a vision-language model, as in DeepMind's 2023 SuccessVQA, which reframes the judgment as the visual question “did the task complete?” and fine-tunes it on Flamingo. It provides a sparse reward, and misjudgments can be exploited by the policy, a failure mode called reward hacking.

ExampleHIL-SERL trains a binary classifier per task on wrist and side-camera images to judge success, and only gives positive reward when it judges success; the paper reports the classifier's accuracy on held-out evaluation generally exceeds 95%.

Also called

Success Classifier, Reward Classifier

Related

Reward Model · VLM-as-Reward · Sparse Reward · Progress Reward Model · Real-World Reinforcement Learning · HIL-SERL

Sources

Du et al. 2023: Vision-Language Models as Success Detectors
Luo et al. 2024: Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning (HIL-SERL)