Behavioral Generalization

A policy still succeeding when a change in the situation forces the 'correct action' itself to change.

This is one dimension of generalization for robot policies. The STAR-Gen taxonomy, proposed in 2025 by Jensen Gao, Dorsa Sadigh, and colleagues, splits manipulation generalization into three categories: visual generalization, where the image changes, such as a new background or lighting; semantic generalization, where the language instruction or concept changes; and behavioral generalization, where the change means even the action an expert should take has to change too. Behavioral generalization covers cases such as: an object's position changes, an object's shape changes so the grasp itself must change, the tabletop becomes cluttered or its height changes, hidden object properties such as mass or friction change, or even the robot itself changes. This kind of variation cannot be handled just by “recognizing” the change — the policy also has to “act” correctly, which requires enough diversity of actions in the training data.

ExampleIn training the cup always sits at the center of the table; at test time it is moved to a corner, or swapped for a cup that can only be picked up by its handle — the policy has to change its trajectory and grasp, which is behavioral generalization. Just changing the tablecloth's color, by contrast, is visual generalization.

Related

Generalization · Visual Generalization · Semantic Generalization · Spatial Generalization · Object Generalization · Cross-Embodiment

Sources

A Taxonomy for Evaluating Generalist Robot Manipulation Policies (STAR-Gen, arXiv 2503.01238)