Energy-Based Reasoners use less compute while matching prior reasoning models
Reported by Hacker News1 source ↓
The researchers report that Energy-Based Reasoners matched or outperformed prior recurrent reasoning models on four algorithmic benchmarks. On Sudoku-Extreme, they achieved 99.9% accuracy using 345 times less inference compute than the next-best model.
Key points
- The model uses a learned energy score and its gradient to refine latent reasoning states and decide when to stop
- It achieved 99.9% accuracy on Sudoku-Extreme
- The researchers report 345 times less inference compute than the next-best model on Sudoku-Extreme
- Across multi-solution benchmarks, it found a larger share of distinct valid solutions while maintaining near-100% single-sample accuracy
Details
The Hacker News post describes Energy-Based Reasoners (EBRs), which guide repeated updates to a model's latent reasoning state using a learned energy function. The energy score evaluates a state, and its gradient directs refinement; a small gradient provides a stopping signal. The method can also select among candidate solutions by choosing the one with the lowest energy.
Across four algorithmic reasoning benchmarks, the researchers report that EBRs matched or outperformed prior recurrent reasoning models while using comparable training compute. On Sudoku-Extreme, they report 99.9% accuracy and 345 times less inference compute than the next-best model. They also report that stopping trajectories when the energy gradient falls below a threshold preserved that accuracy while using 10.9 times fewer FLOPs than a fixed-depth EBR.
Written from coverage as of
Images
- Hacker News