Ablation Study

消融实验Common

Removing or swapping one component of a method to see how performance changes, testing whether it matters.

An ablation study starts from a complete method and removes or replaces one component at a time — a module, a loss term, an input modality, a data source, or a training trick — then retrains and re-evaluates under the same settings and compares the change in performance, to judge how much each design choice actually contributes. The term is borrowed from biology, where removing part of a tissue is used to study its function; according to Wikipedia, the AI pioneer Allen Newell used it in a 1974 speech-recognition tutorial. It's a paper's main evidence for explaining why it was designed a certain way: if removing a module barely changes performance, that suggests it isn't essential. Other variables need to be held fixed during an ablation, and enough trials need to be run for the comparison, or the difference observed may just be noise.

ExampleThe ACT paper's ablations separately remove action chunking, temporal ensembling, and CVAE training, finding that when training on human demonstration data, removing the CVAE clearly hurts success rate.

Also called: ablation
Related: Baseline · State of the Art (SOTA) · Success Rate · Action Chunking · Temporal Ensembling
Sources: Wikipedia: Ablation (artificial intelligence)
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (ACT)