Progress Score

A metric that scores how many steps of a task a robot completed, rather than just recording binary success or failure.

Progress score is a class of evaluation metric: a task is broken into stages or sub-goals ahead of time, and a robot earns credit for each stage it completes, usually normalized to a 0–1 or 0–100 scale. It complements binary success rate. On long-horizon, difficult tasks, every policy's success rate can sit near 0, which makes it impossible to tell them apart; scoring by progress reveals which policy got further and where exactly it got stuck. The downside is that the scoring rubric is defined by whichever researchers wrote it, so scores are not directly comparable across different papers. Physical Intelligence's π0 paper designed a separate scoring rubric for each task; HomeRobot OVMM uses partial success, awarding 1 point per completed stage, as one basis for its ranking; and RoboArena has evaluators assign a 0–100 progress score to each trial.

ExampleIn the π0 paper's laundry-folding task, taking a garment out of the basket, laying it flat, folding it, and stacking it neatly each earn 1 point; completing all of them is a perfect score, while only laying it flat earns half credit.

Also called

Task Progress, Partial Success, Partial Credit

Related

Success Rate · Evaluation Protocol · Long-horizon Task · Real-World Evaluation · RoboArena · Average Length (CALVIN)

Sources

π0: A Vision-Language-Action Flow Model for General Robot Control (arXiv 2410.24164)
HomeRobot: Open-Vocabulary Mobile Manipulation (arXiv 2306.11565)
RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies (arXiv 2506.18123)