[Paper] Think Less, Locally: Local Preference Learning for Reasoning Compression
Reported by r/LocalLLaMA1 source ↓
A paper introduces LoCo, which trains models to decide when to stop reasoning. Its authors report shorter outputs with maintained or improved accuracy across math and code benchmarks.
Key points
- LoCo reduced completion length by 41.7%–45.4% on math benchmarks and 23.4%–36.3% on code benchmarks
- The authors report maintained or improved accuracy and the best AUC across all code benchmarks
- Training data came from self-generated math traces; evaluation covered five math and three code benchmarks
- The paper, code and a 7B model are available on arXiv, GitHub and Hugging Face
Sources
Details
The paper describes LoCo as a method for identifying points in a reasoning trace where further thought may be unnecessary. It freezes the reasoning prefix at those points, generates shorter verified continuations, and uses the resulting preference pairs to train the model with DPO.
The authors report that LoCo improved the accuracy-efficiency trade-off on math tasks and transferred to code generation despite using math-only training data. The post links to the paper, code repository and LoCo-7B model.
Written from coverage as of