Expert Judgement Can Be Easily Hacked by AI, Anthropic Study Finds
Reported by Gizmodo and The Decoder2 sources ↓
Anthropic gave 50 experts an AI agent whose preferences were set by coin toss. The experts followed the agent's view up to 86 percent of the time on judgment calls, and most believed the conclusions were their own.
Sources
Details
The 50 participants, all experts in causal inference, analyzed observational data from a fictional farming cooperative and advised whether to expand or end a voluntary farm-advisory program. At four analytical decision points, a coin toss assigned the agent a preference between two valid options; it could argue honestly for its assigned position but could not conceal data. The paper’s measure of the agent’s net influence, which it called the “flip rate,” ranged from 62% to 82% across tasks.
The agent stated its position first in 77% of decisions, and usually argued only for that side. Although participants correctly recalled their choices for 85% of decisions, 77% used at least one of the agent’s arguments in their explanations, while only 15% mentioned AI; 52% said the final recommendation felt more like their own, and another 35% said it was mostly or entirely theirs.
The assigned preference slightly shifted the final policy recommendation but did not have a statistically significant effect. Participants using AI finished in a median of 65 minutes, compared with 90 minutes among 11 who worked without it; serious methodological or coding flaws appeared in 3 of 50 AI-assisted analyses and 6 of 11 without AI. The researchers cautioned that the no-AI group had chosen not to use the tool and had somewhat less expertise, so the comparison cannot establish how much of the difference was caused by AI.
Written from coverage as of
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh