Synthetic Data

Training data manufactured by computers, via simulation or generative models, instead of collected from the real world.

Synthetic data is data produced by a computer rather than collected from reality — NVIDIA defines it as text, images, and video generated using computer simulation, generative AI models, or a combination of both. In embodied AI there are mainly three routes: rendering and executing tasks inside a simulator (simulation data); automatically transforming and expanding a small number of human demonstrations into many new ones, as in MimicGen; and directly generating videos of a robot working with a video-generation model or world model, then filling in pseudo action labels with an inverse dynamics model (a model that infers actions from before-and-after frames), as in NVIDIA's DreamGen. It helps relieve the scarcity of real-robot data, but whether the generated content is physically plausible and close to the real distribution ultimately still has to be checked with real-robot evaluation.

ExampleMimicGen automatically expands fewer than 200 human demonstrations into more than 50,000, covering 18 tasks; DreamGen, using only teleoperation data from a single pick-and-place task, taught a humanoid robot 22 new skills across 10 new environments.

Also called

Generated Data

Related

Simulation Data · MimicGen · DreamGen · Neural Trajectories · Pseudo Action Labels · Generative Data Augmentation

Sources

NVIDIA Glossary: What Is Synthetic Data Generation?
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
DreamGen: Unlocking Generalization in Robot Learning through Video World Models (NVIDIA GEAR)

As of

2025-05