Intervention Rate

How often a human has to take over or correct a policy while it runs autonomously; lower means more independent operation.

Intervention rate measures how autonomous a system really is: the fraction of the time a human has to take over or correct a robot or self-driving system while it operates, usually counted per episode, per hour, or per kilometer, and sometimes reported instead as the average time between interventions (mean time between interventions). The concept comes from the “takeover” or “disengagement” metric used in self-driving cars, and it matters just as much for deployed robots: success rate alone only tells you the outcome, while intervention rate reveals how much human labor was needed to get there, and directly determines how many robots one person can supervise at once. Human-in-the-loop methods such as HG-DAgger and Sirius have a person take over when the robot makes a mistake or a situation gets risky, and feed those intervention episodes back into training as correction data, so the intervention rate should fall as more deployment rounds accumulate. Because different teams may define what counts as one intervention differently, reports need to state that definition, or the numbers are not comparable across teams.

ExampleSirius lets a human remotely take over whenever the robot makes a mistake and adds that intervention data back into training; as more rounds of deployment accumulate, the amount of human intervention needed drops sharply, and by the third round the robot operates autonomously most of the time.

Also called

Takeover Rate, Disengagement Rate

Related

Mean Time Between Interventions · Human-in-the-Loop · Human-Gated DAgger · Human Intervention Data · Remote Teleoperation Takeover · Levels of Autonomy

Sources

HG-DAgger: Interactive Imitation Learning with Human Experts (arXiv 1810.02890)
Sirius: Robot Learning on the Job (project page)