A new way of measuring AI models suggests China's threat to OpenAI and Anthropic may have been overblown
Reported by Business Insider1 source ↓
Artificial Analysis's revamped tests put leading open-weight models further behind top closed models, raising questions about claims that Chinese AI could challenge US companies. The new tests include private tasks and changed scoring.
Key points
- Artificial Analysis's new ranking widened the gap between top closed and open-weight models
- Open-weight leaders scored 32 points, versus 58 for the overall leader, Claude Opus 5.5
- The new system keeps many tests private and changed the tasks and their weighting
- Open-weight models remain competitive in software development, according to researcher Nathan Lambert
- OpenRouter traffic represents only part of demand for open-weight models
Details
Business Insider reports that Artificial Analysis overhauled its model evaluations, changing the tasks and how they are weighted while keeping many tests private. Under the old system, the best open-weight models trailed the overall leader by three points; under the new system, the gap was 12 points. The report says open-weight leaders scored 32 points, compared with 58 for Anthropic's Claude Opus 5.5, then the overall leader.
The results suggest that earlier tests may have overstated open-weight models' performance, in part because companies could train for known benchmarks. Nathan Lambert, cofounder of Trillium Labs, said the previous index was too favorable to open-weight models. Business Insider also reports that even Anthropic's smaller Claude Haiku 5.5 scored above every model with publicly released weights.
OpenRouter captures only part of the market, so those figures do not establish total demand. Lambert said open-weight models remain competitive in software development and can give businesses more control and potentially lower costs.
Written from coverage as of
Images
- Business Insider
Stocks in this story
Change since open: 20 Nov to 21 Nov 2026
Last refresh