Neural Motion Planning

Using a neural network trained on huge numbers of planning examples to learn to generate collision-free motion directly.

Neural motion planning uses a neural network to perform or assist motion planning: given a scene observation (often a point cloud) and the current and goal configuration, it outputs a collision-free path or the next action. Traditional sampling-based planners (like RRT) and trajectory optimizers restart from scratch on every new problem, sometimes taking minutes on cluttered scenes, and they depend on an accurate geometric model of the scene. Neural approaches instead first use a traditional planner to generate large numbers of expert solutions in simulation, then distill them into a network with imitation learning, so an action comes out with just a forward pass at inference time. Representative work includes 2018's MPNet (which encodes an obstacle point cloud and predicts the next configuration step by step, combinable with RRT* for a success guarantee), NVIDIA and University of Washington's 2022 MπNets (trained on over 500,000 environments and 3 million planning problems, acting directly from a depth camera's point cloud), and CMU's 2024 Neural MP (which procedurally generates huge numbers of scenes to train a general policy, adding a lightweight optimization step at deployment for safety). The difficulty is that the network's raw output carries no collision-free guarantee, so it is usually paired with a collision check or a fallback mechanism.

ExampleNeural MP was tested on 64 tasks across 4 real-world environment types, generating arm motion directly from scene point clouds, with success rates 23, 17, and 79 percentage points higher than sampling-based, optimization-based, and learning-based baselines respectively.

Also called

Learning-Based Motion Planning

Related

Motion Planning · Motion Policy Networks · Rapidly-exploring Random Tree · Imitation Learning · Collision Checking · cuRobo (NVIDIA GPU-accelerated motion planning)

Sources

Neural MP: A Generalist Neural Motion Planner (arXiv 2409.05864)
Motion Policy Networks (arXiv 2210.12209)
Motion Planning Networks (arXiv 1806.05767)

As of

2024-09