SERL

An open-source real-world reinforcement learning toolkit from Berkeley and others that can train a policy on a real robot arm in tens of minutes.

SERL was released by researchers at UC Berkeley, Stanford, the University of Washington, and Intrinsic (Jianlan Luo, Sergey Levine, and others) in January 2024, published at ICRA 2024. Getting reinforcement learning to work on real robots is hard not just because of the algorithms, but because of engineering problems: how to define the reward, how to reset after each episode, and whether the low-level controller is safe. SERL packages solutions to these as open-source tools: RLPD, a sample-efficient off-policy algorithm that can also make use of a small number of human demonstrations; an image classifier that judges task success and serves as the reward; two alternating 'forward' and 'backward' policies that reset the environment automatically, removing the need for manual resets; and an impedance controller for the Franka arm that keeps contact safe and compliant. In the paper's experiments, tasks like PCB component insertion, cable routing, and object relocation reach close to 100% success after just 25 to 50 minutes of training. The follow-up work HIL-SERL adds live human corrections on top of this.

ExampleFor PCB component insertion, a small number of human demonstrations are recorded first, then an image classifier that judges 'is the component seated correctly' is trained as the reward; the arm then practices on its own and reaches reliable insertion in under an hour.

Also called

Sample-Efficient Robotic Reinforcement Learning, SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning

Related

HIL-SERL · Real-World Reinforcement Learning · Reinforcement Learning with Prior Data · Success Detector · Impedance Control · Sample Efficiency

Sources

SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning (arXiv 2401.16013)
SERL project page

As of

2024-05