The Bitter Lesson

Rich Sutton's argument that, over time, general methods that scale with compute beat hand-engineered domain knowledge.

This is a short essay published in March 2019 by Rich Sutton, one of the founders of reinforcement learning. Looking back over 70 years of AI research, he argues that what has actually worked in the long run is general-purpose methods that can fully exploit growing compute — search and learning — rather than hard-coding human understanding of a domain into the system. Chess, Go, speech, and vision all followed the same pattern: hand-engineered knowledge produced short-term progress, but was ultimately overtaken by scaled-up search and learning, because the cost of a unit of compute keeps falling exponentially. In embodied AI, this essay is often invoked to support an “end-to-end plus big data plus large models” approach and to argue against over-engineered, hand-designed modules; the point of contention is that robot data is far scarcer than text and images, so whether the same lesson applies is still debated.

ExampleSutton's own example: computer Go long relied on hand-crafted Go theory and Go-specific structure, and was eventually surpassed completely by methods based on large-scale search plus self-play learning of a value function.

Related

Scaling Law · End-to-End · Moravec's Paradox · Foundation Model · Data Scarcity · Embodied AI

Sources

The Bitter Lesson (Rich Sutton, 2019)