Causal Confusion (Causal Misidentification)

Imitation learning latching onto a cue correlated with the expert's actions but not actually its cause, then failing at deployment.

Causal confusion was systematically identified by de Haan, Jayaraman, and Levine in a NeurIPS 2019 paper. Behavior cloning treats imitation as supervised learning, regressing directly from observation to the expert's action, learning only correlation without distinguishing cause from effect; if some cue in the training data happens to correlate strongly with the expert's actions, the model will latch onto it. That cue is present the whole time during training, so the loss stays low; at deployment, the states the policy wanders into differ from the expert's (distribution shift), the cue stops being reliable, and the policy fails. A counterintuitive symptom: giving the model more input information can actually make performance worse, especially common when the input includes history. The paper proposes targeted interventions — trying an action in the environment or asking the expert — to recover the correct causal structure, outperforming baselines like DAgger. It's a reminder that more input isn't automatically better for a robot policy.

ExampleThe paper's driving example: model A sees the full dashboard view, model B has the dashboard masked out. Both reach low training loss, but on the road B drives well and A doesn't — the dashboard has a brake-light indicator that lights up whenever the brake is pressed, and A learned “brake when the light is on,” mistaking an effect of braking for its cause.

Also called

Shortcut Learning (in Imitation Learning), Causal Misidentification

Related

Behavior Cloning · Imitation Learning · Distribution Shift · Compounding Error · DAgger · Overfitting

Sources

Causal Confusion in Imitation Learning (arXiv 1905.11979)