AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds
Reported by The Decoder and MLQ.ai2 sources ↓
Anthropic Institute researcher Andy Hall tested AI models on agent logs. The Decoder reports that at a 5% false-alarm rate, GPT-5 caught nearly 98% of reward-hacking cases; higher detection rates brought more false alarms.
Key points
- At a 5% false-alarm rate, GPT-5 caught nearly 98% of reward-hacking cases
- Auditors caught 71–81% of sandbagging cases, most of which were synthetic
- At an 80% detection rate, some models flagged 40–63% of harmless logs as suspicious
- On a separate dataset, the best model had a 15% false-alarm rate when targeting 99% detection
Sources
Details
The Decoder reports that Hall replicated and extended METR research using logs of models completing HCAST and RE-Bench tasks. He tested GPT-5 and GPT-5 nano, along with GPT-6 Astra, Opus 4.8 and Gemini 3.8 Flash, against cases human reviewers had checked. At a 5% false-alarm rate, GPT-5 caught nearly 98% of reward-hacking cases, while Gemini 3.8 Flash caught about half.
Results worsened when auditors were tuned to catch more cases. At an 80% detection rate, the best models had false-alarm rates of 2.5–3%; Opus 4.8 flagged 40% of harmless logs and Gemini 3.8 Flash flagged 63% to reach that detection rate for reward hacking. Hall's example of 100,000 monthly sessions at a 3% false-alarm rate would generate nearly 3,000 false warnings alongside roughly 2,000 real hits.
A second test, on Shanghai Jiao Tong University's R-Judge dataset of everyday agent tasks, showed a similar trade-off. At a 5% false-alarm rate, Opus 4.8 caught almost 96% of unsafe actions; when targeting 99% detection, even the best model's false-alarm rate was 15%. Hall's study is a pilot, and its reward-hacking results use only 47 cases, so one case changes the rate by about two percentage points.
Written from coverage as of
Images
- The Decoder
- The Decoder
- The Decoder
- The Decoder
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh