Stop-Gradient

A value is used normally in the forward pass but treated as a constant during backpropagation, blocking its gradient.

Stop-gradient is a basic deep-learning operation: some quantity is used normally in the forward computation, but during backpropagation it is treated as a constant, so no gradient flows back through it to earlier parameters. Papers often write it as sg(·); PyTorch uses .detach(), and JAX uses jax.lax.stop_gradient. It has many uses: in reinforcement learning it stabilizes temporal-difference learning by blocking gradients into the target value; VQ-VAE uses it so the codebook and the encoder can be updated separately; and SimSiam found it to be the key ingredient that stops self-supervised learning's representations from collapsing. In VLAs, Physical Intelligence's 2025 knowledge insulation blocks the gradient flowing from the action expert back into the VLM backbone, so a newly initialized action expert doesn't disrupt the backbone's pretrained knowledge, and the backbone instead learns from discretized action tokens. Note this is not the same as gradient clipping, which limits gradient magnitude rather than blocking it.

ExampleIn knowledge-insulation training, the continuous action expert reads features from the VLM backbone to generate actions, but its loss is never backpropagated into the backbone; the backbone adapts to robot data only through the next-token-prediction loss on discretized action tokens.

Also called

stop-grad, detach, sg(·)

Related

Knowledge Insulation · Target Network · Representation Collapse · Backbone Freezing · Backpropagation · Vector-Quantized Variational Autoencoder

Sources

Chen & He 2020: Exploring Simple Siamese Representation Learning (SimSiam)
Driess et al. 2025: Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better

As of

2025-05