Learning-Based Control

Using data and machine learning to build or improve a controller, rather than relying entirely on hand-derived equations and tuning.

A broad term for methods that use learned data to construct or improve a controller, as opposed to model-based control, which relies on hand-built models. Three approaches are typical: learning a dynamics model (or its error) and handing it to a traditional controller such as an MPC; training a neural-network policy end to end with reinforcement learning or imitation learning, mapping observations directly to joint targets or torques; and online learning that adjusts a controller's parameters as it runs. A 2022 survey by Brunke et al. in Annual Review of Control, Robotics, and Autonomous Systems groups methods that learn uncertain dynamics to safely improve performance under ‘learning-based control,’ contrasting it with safe reinforcement learning. It suits hard-to-model effects like contact, friction, and motor behavior, at the cost of needing large amounts of data or simulation, and it is difficult to give strict stability guarantees. Today, most quadruped and humanoid locomotion controllers use reinforcement-learning policies trained in simulation.

ExampleHwangbo et al., published in Science Robotics in 2019, first trained an actuator network on real-robot data to model motor behavior, then trained an ANYmal quadruped's policy with reinforcement learning in simulation using that network; after transfer to the real robot it ran faster than prior methods and could get back up on its own after falling from awkward poses.

Also called

Data-Driven Control

Related

Model-Based Control · RL-based Locomotion Control · Reinforcement Learning · Imitation Learning · Safe Reinforcement Learning · Learning-Based Whole-Body Control

Sources

Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning (Brunke et al., arXiv 2108.06266)
Learning Agile and Dynamic Motor Skills for Legged Robots (Hwangbo et al., Science Robotics 2019, arXiv 1901.08652)