Score Matching

Training a network to estimate the gradient of a data distribution's log-density, without needing its normalizing constant.

The score is the gradient of the log probability density with respect to the input, ∇x log p(x), and it points toward where data is more likely to occur. Hyvärinen proposed score matching in 2005: train a model's score to fit the data's score, which sidesteps the normalizing constant that is hard to compute in energy-based models. Later, denoising score matching turned the problem into “add noise to data, then learn to denoise it.” In 2019, Song and Ermon used score estimation under multiple noise levels plus Langevin dynamics sampling for image generation; in 2020, Song and colleagues unified score-based models and diffusion models using stochastic differential equations, where the reverse denoising process depends only on the score at each noise level. DDPM's noise prediction and flow matching's velocity prediction can both be converted into score estimates, so the training behind generative robot policies such as Diffusion Policy is, underneath, score matching.

ExampleDiffusion Policy trains by adding Gaussian noise of varying strength to expert actions and having the network predict the noise that was added; dividing that prediction by the noise's standard deviation and negating it gives a score estimate for the noisy-action distribution.

Also called

Score Function, Denoising Score Matching

Related

Diffusion Model · Denoising Diffusion Probabilistic Model · Denoising Loss (Diffusion Loss) · Flow Matching · Energy-Based Model · Prediction Target Parameterization

Sources

Hyvärinen 2005: Estimation of Non-Normalized Statistical Models by Score Matching (JMLR)
Song & Ermon 2019: Generative Modeling by Estimating Gradients of the Data Distribution
Song et al. 2020: Score-Based Generative Modeling through Stochastic Differential Equations