Learned State Estimator

A neural network that estimates quantities like body velocity from joint and IMU data, often trained jointly with a locomotion policy.

A learned state estimator uses a neural network to replace or supplement a Kalman filter, regressing hard-to-measure states directly from proprioceptive data — a history of joint angles, joint velocities, and IMU readings. A landmark example is a 2022 RA-L paper from Hwangbo’s group at KAIST in Korea: it trains a control policy and an estimator network simultaneously in simulation, the policy with PPO reinforcement learning and the estimator with supervised learning against simulation ground truth, estimating body linear velocity, foot height, and contact probability, which then feed into the policy as input. This needs no predefined gait and no foot-contact sensor, and the estimator sees plenty of randomized slippery and rough terrain during training. In the paper, the quadruped robot reached a top speed of 3.75 m/s on flat ground and 3.54 m/s on a wet surface with a friction coefficient of 0.22. Many later legged reinforcement-learning locomotion projects have adopted a similar estimator network.

ExampleIn Ji and colleagues’ real-robot system, the estimator network first estimates body linear velocity, foot height, and contact probability from a history of proprioception, and the policy network then outputs desired joint positions based on this, completing high-speed walking over slopes, wet boards, and bumpy ground.

Also called

Concurrent Estimator Network, Estimator Network

Related

State Estimation · RL-based Locomotion Control · Privileged Information · Leg Odometry · DreamWaQ · HIM

Sources

Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion (Ji et al., RA-L 2022, arXiv 2202.05481)