The inventor of the bitter lesson has a bitter lesson for LLM scaling enthusiasts
Reported by The Decoder1 source ↓
Turing Award winner Richard Sutton says raw compute alone is not enough for intelligence and argues that AI needs new learning methods. Researchers disagree over whether continued LLM scaling is sufficient.
Key points
- Sutton says AI needs learning methods beyond simply adding compute to model training
- He argues agents should learn continuously from experience without losing old knowledge
- Jerry Tworek says the algorithm matters more as compute for training increases
- Sam Altman and Dario Amodei argue that LLM scaling has further room to improve
Sources
Details
In a recent post, Richard Sutton, the Turing Award-winning author of the 2019 essay “The Bitter Lesson,” said AI science is far from finished and that raw compute alone is not enough for intelligence. Sutton argues that progress requires new methods, not only larger training runs.
Sutton has previously criticized reliance on synthetic data and human expertise, arguing that simulated data captures only a small part of the complexity of the real world. He advocates agents that use world models and continue learning from their own experience without losing earlier knowledge; he says today’s language models stop learning after training.
AI researcher Jerry Tworek said the bitter lesson still applies, but that the algorithm becomes more important as more compute is used. Jayden Teoh said researchers can misread the lesson by dismissing ideas that do not scale indefinitely, even when those ideas may contribute to progress.
Anthropic CEO Dario Amodei has also argued that scaling progress can continue. The article notes that both executives lead companies whose technical approaches rely on scaling.
Written from coverage as of
Images
- The Decoder
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh