Data Pyramid

A three-tier framework organizing robot training data by volume and how closely it matches a real robot.

The data pyramid is a framework NVIDIA used to organize training data in its March 2025 GR00T N1 paper: the base is web data and human video, by far the largest in volume; the middle tier is data generated by physical simulation or synthesized by neural networks such as video-generation models; and the top tier is data collected on real robots. Moving up the pyramid, data volume shrinks while how closely it matches the specific embodiment (a robot's physical hardware) grows. The base supplies general common sense and behavioral priors, while the top ensures actions can actually be executed on the real robot. It targets the reality that real-robot data is expensive and scarce: build on a large, cheap base and calibrate with a small amount of real-robot data at the top.

ExampleGR00T N1's base tier used human video such as Ego4D and EPIC-KITCHENS; its middle tier had about 827 hours of neural trajectories generated by a video model plus 780,000 DexMimicGen simulated trajectories; its top tier was real-robot data such as Fourier GR-1 teleoperation data, Open X-Embodiment, and AgiBot World.

Also called

Robot Data Pyramid

Related

Real-Robot Data · Simulation Data · Synthetic Data · Human Video Data · Neural Trajectories · NVIDIA Isaac GR00T N1

Sources

GR00T N1: An Open Foundation Model for Generalist Humanoid Robots (arXiv 2503.14734)

As of

2025-03