Mean Squared Error

The average of the squared difference between predictions and ground truth; the most common loss for regression.

Mean squared error squares the difference between each sample's prediction and its ground truth, then averages over all samples; the sum without averaging is often called L2 loss. It's differentiable everywhere, and its gradient shrinks as the error shrinks, giving smooth optimization, which makes it the default loss for regression tasks. From a probabilistic view, minimizing MSE is equivalent to maximum likelihood estimation under the assumption that errors are Gaussian, with the optimal solution being the conditional mean. Because it squares the error, large errors get amplified, so MSE is more easily thrown off by outliers than L1 loss. In robot imitation learning it has a well-known failure mode: if a demonstrator sometimes goes around an obstacle from the left and sometimes from the right, regressing actions directly with MSE learns the average of the two, producing a path down the middle that matches neither — this is the action-multimodality problem, and it's part of why generative policies like diffusion policies and flow matching became popular. A diffusion model's own training objective is also MSE, just applied to the noise that was added rather than to the action itself.

ExampleDiffusion Policy adds random noise to an expert action sequence during training and has the network predict that added noise; the loss is the MSE between predicted and true noise, and the paper notes that minimizing this loss also minimizes a variational bound on the KL divergence between the data distribution and the model's distribution.

Also called

MSE, L2 Loss, Squared Error Loss

Related

L1 Loss · Loss Function · Maximum Likelihood Estimation · Action Multimodality · Continuous Action Regression · Denoising Loss (Diffusion Loss)

Sources

Linear regression: Loss（Google Machine Learning Crash Course）
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (arXiv 2303.04137)