Quick Take
- In Anthropic’s study, 50 professional analysts went with their AI agent’s lean 86% of the time, even though a coin flip had set that lean.
- 77% later explained their choices with the agent’s arguments, but only 15% gave it credit.
- It is one fictional task, and the agents’ preferences were made stronger on purpose, so treat the numbers as a warning sign, not a law of nature.
Who is really making the call when an expert works with an AI agent? Anthropic tried to measure it, and the results are uncomfortable.
The experiment
Anthropic asked analysts with experience in causal inference, the work of figuring out from data whether one thing causes another, to do a realistic analysis with Claude Code. The client was a made-up farming cooperative deciding whether its advisory program worked and should be expanded.
Along the way, each analyst faced four choices with two reasonable answers each. One example: should productivity be judged by crop yield per hectare, or by yield per unit of operating cost? Then they made a final call on the program.
Here is the twist. Each agent was set up to favor one option on each decision, and a coin flip picked which one. The agent was told to make its best honest case, change its mind if the evidence demanded it, and never mislead. The analysts were not told about the lean.
What happened
If the agent had no influence, analysts would match its lean about half the time. Instead, they matched it 86% of the time across the four choices.
Then came the explanations. When asked why they chose what they chose, 77% used the agent’s own arguments, sometimes word for word. Only 15% credited the agent. Analysts who followed the agent closely were about as confident as everyone else, and roughly as likely to call the result their own judgment.
The limits Anthropic admits
Credit to Anthropic for flagging its own caveats. Per the study write-up:
- The agents’ preferences were strengthened on purpose to measure their effect. Real agents, without clear leanings and instructions to voice them, may push less.
- Anthropic says it does not know how much this tells us about the real world.
- People who worked without the agent made far more serious errors and were slower, but that group was small and less experienced, so the comparison is loose.
It is also one fictional farming task.
Fact, claim, judgment
Fact: under these conditions, analysts went with a lean set by coin flip on 86% of the four choices, and only 15% credited the agent, per Anthropic’s study.
Claim: that an AI’s arbitrary or misaligned preferences could quietly steer research and policy. Anthropic raises this as a risk. It is a reasonable worry, not a measured result.
Judgment: the scary headline number is inflated by design. The quieter finding matters more: people repeated the agent’s arguments as their own and felt just as sure. That makes “a human signed off” a weaker safety check than it sounds. Anthropic notes that human review works as an independent check only when the reviewer’s view is truly their own, which matters as labs use agents in safety research.
What you can do
If you use an agent for judgment calls, write down your own choice first. Ask the agent to make the best case for the option it did not pick. And credit the tool when its argument wins, so you and your readers know where the idea came from.
Bandwagon Check
Fairly hyped on the 86% headline; not overhyped, and if anything underhyped on what it means for human review. The 86% figure is boosted by a deliberately pushy agent. But the core finding, that experts adopt an AI’s views and then present them as their own, gets too little attention and matters for anyone who treats human review as a safeguard.
Sources
By Willa Ravenwire · The AI Bandwagon