[Paper] Tree-Ising: Repairing Language Models with Bit-Level Ising Edits
Reported by r/LocalLLaMA and Blockchain News2 sources ↓
Tree-Ising edits BF16 model weights to reduce reasoning loops. The researchers report 54.6% accuracy on MATH-500 for Qwen3.5-0.8B, versus 51.7% for an agent-made repair and 7.9% for the base model.
Key points
- On Qwen3.5-0.8B, it scored 54.6% on MATH-500, compared with 51.7% for the agent repair
- The researchers report raising Qwen3.5-2B's GSM8K accuracy from 21.6% to 69.4%
- The repair uses four forward-pass requests and zero LLM tokens; GPU time for annealing was about $0.90
- The researchers report a proof-of-concept repair subproblem run on an IBM Heron processor
Sources
Details
Tree-Ising treats reasoning-loop repair as a sparse discrete optimization problem over a model’s BF16 weights. The researchers say its allowed edits are sign flips and exponent shifts that preserve shared group exponents and mantissas. They divide the optimization into independent problems for each output row and solve them with GPU-based simulated annealing or simulated bifurcation.
On Qwen3.5-0.8B, the researchers report 54.6% accuracy on MATH-500 for the annealing repair, compared with 51.7% for the earlier GPT-6 Astra agent repair and 7.9% for the unedited model. Simulated bifurcation scored 55.1%, which the researchers describe as statistically indistinguishable from the annealing result. They report improvements over the agent edit across five math benchmarks.
The researchers report that Tree-Ising required four forward-pass requests and no LLM tokens; the annealing run cost about $0.90 in GPU time. On Qwen3.5-2B, they report increasing greedy GSM8K accuracy from 21.6% to 69.4%, with layer and dose selection based only on the base model. They also report modifying 13% to 17% fewer weights and making 2.2 to 2.3 times fewer BF16 bit flips than prior approaches.
The paper appeared on arXiv on November 10, 2026, according to Blockchain News. The researchers also report running an eight-variable repair subproblem with QAOA on an IBM Heron processor.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh