Noise Schedule

The timetable specifying how much noise a diffusion model adds at each step, which affects training and generation quality.

The noise schedule is a design choice in a diffusion model: it specifies, at step t of the forward noising process, how much of the original data remains and how much is noise. 2020's DDPM used a linear schedule; in 2021, OpenAI's Nichol and Dhariwal found that on low-resolution images this wastes steps, since the later steps are already almost pure noise, and proposed a cosine schedule instead, which removes information at a steady rate through the middle and changes more gently at both ends. The schedule determines how much training gets allocated to each noise level and noticeably affects generation quality; in flow matching, the corresponding design choice is the sampling distribution over training timesteps. Robot policies need this tuning too: Diffusion Policy uses a cosine schedule, and π0 deliberately oversamples high-noise timesteps when training with flow matching.

ExampleThe Diffusion Policy paper compares options and settles on the cosine schedule proposed by iDDPM, training with 100 diffusion steps but running only 10 DDIM steps at inference on the real robot, producing an action roughly every 0.1 seconds on an RTX 3080.

Also called

Noise Scheduler

Related

Diffusion Model · Denoising Diffusion Probabilistic Model · Denoising Steps · Flow Matching · Diffusion Policy · Prediction Target Parameterization

Sources

Improved Denoising Diffusion Probabilistic Models (arXiv:2102.09672)
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (arXiv:2303.04137)
π0: A Vision-Language-Action Flow Model for General Robot Control (arXiv:2410.24164)