Simulation Data

Training data generated automatically by having a virtual robot perform tasks inside a physics simulator.

Simulation data is generated inside a physics simulator such as Isaac Sim or MuJoCo: the scene, objects, and robot are all virtual, and a script, motion planner, or reinforcement-learning policy completes tasks on its own while images, depth, joint state, and actions are all recorded. The advantages are that it's cheap, runs at massive parallel scale, comes with perfectly accurate labels for free, and can randomize lighting, materials, and object placement — domain randomization — to cover long-tail situations that are hard to capture with a real robot. The main obstacle is the sim-to-real gap: simulated contact physics and rendered images never perfectly match reality, and deformable objects and fine-grained contact are especially hard to get right. A common approach is to pretrain at scale on simulation data first, then fine-tune with a small amount of real-robot data, or train on a mixture of both together.

ExampleTeams such as Galbot pretrained GraspVLA using SynGrasp-1B, a billion-frame simulated grasping dataset (over 10,000 objects across 240 categories), for all of its action data, combined with internet image-text grounding data for joint pretraining; without using any real-robot data at all, it transferred zero-shot to real-world grasping.

Also called

Sim Data

Related

Simulator · Synthetic Data · Sim-to-Real Gap (Reality Gap) · Domain Randomization · Sim-to-Real Transfer · SynGrasp-1B

Sources

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations

As of

2025