In-distribution

A test-time situation drawn from the same distribution as the training data — something the model has effectively seen before.

Machine learning typically assumes training and test data come from the same probability distribution; a test sample satisfying this is called in-distribution (ID), and one from a different distribution is out-of-distribution (OOD). A 2021 OOD-detection survey by Jingkang Yang and colleagues splits distribution change into two kinds: covariate shift, where the input's appearance changes — different lighting, background, or camera — but the task categories stay the same, and semantic shift, where entirely new categories not seen in training appear. In robotics, “in-distribution” usually means the objects, scene, placement, and instructions at test time all fall within the range covered by the training demonstrations. Many policies have a high success rate in-distribution but drop sharply the moment the tablecloth or the object changes, so papers often report in-distribution and out-of-distribution results separately as a way to measure generalization. This concept connects directly to generalization, out-of-distribution, distribution shift, and the long-tail problem.

ExampleA robot arm is trained with 50 demonstrations to put a red block on a plate; if the test still uses the same table, the same red block, and a position within the training range, that is an in-distribution test — swapping in an unseen green cup would be out-of-distribution.

Also called

ID

Related

Out-of-Distribution · Generalization · Distribution Shift · Long-tail Problem · Robustness · Object Generalization

Sources

Generalized Out-of-Distribution Detection: A Survey (Yang et al.)