Optimal Control

Finding, subject to the system's dynamics, a sequence of control inputs that minimizes total cost.

Optimal control is a branch of control theory studying how to choose control inputs over a time span so an objective function is minimized. The standard form is min J = φ(x(T)) + ∫ L(x, u) dt, subject to ẋ = f(x, u): x is the state (e.g., joint angles and velocities), u the control input (e.g., torque), L the running cost (deviation from the goal, energy use), φ the terminal cost, and f the dynamics model. Its theoretical foundations are Pontryagin's maximum principle and Bellman's dynamic programming, both from the 1950s. The special case of linear dynamics with quadratic cost is the LQR, whose solution is linear feedback u = −Kx. Trajectory optimization, iLQR/DDP, and model predictive control (which re-solves a finite-horizon optimal control problem every cycle) in robotics all fall under this umbrella; reinforcement learning can be seen as solving the same class of problem when the model is unknown.

ExampleBaidu Apollo's autonomous-driving lateral controller uses LQR to compute steering: it weights ‘deviation from the reference trajectory’ and ‘steering effort’ into a cost, then finds the feedback gain that minimizes total cost.

Also called

Optimal Control Problem, OCP

Related

Linear Quadratic Regulator · Model Predictive Control · Trajectory Optimization · Iterative Linear Quadratic Regulator · Differential Dynamic Programming · Reinforcement Learning

Sources

Wikipedia: Optimal control
Underactuated Robotics（Tedrake）: Linear Quadratic Regulators
Apollo Control 模块说明（README_cn） (Chinese)