OpenAI rates GPT-6.2 Astra High for self-improvement despite missing two test cutoffs
Reported by NewsBytes, Hacker News, The Defiant and 11 more outlets14 sources ↓
Part of a developing story. How it developed ↓
OpenAI says GPT-6.2 Astra is its first externally deployed model to meet its High threshold for AI self-improvement. Its system card reports scores below the cutoffs on two second-stage tests and does not disclose how the five evaluations were weighted.
Key points
- Astra scored 42.87% on NanoGPT, below its 80% cutoff
- Astra scored 74.79% on PostTrainBench Lite, below its 90% cutoff
- Only MLE-Bench Revised cleared its second-stage cutoff; Sol also cleared it
- The system card gives no numeric cutoffs for the two first-stage tests
- GPT-6.2 Sol and GPT-6 Astra were not rated High
Sources
- NewsBytesRead ↗OpenAI releases GPT-6.2 Astra, Sol to outshine Anthropic's Claude Fable
- Hacker NewsRead ↗GPT‑6.2 Astra and Sol
- The DefiantRead ↗GPT-6.2 Models Rise on AI Index, but Coding Scores Stall
- noteRead ↗GPT-6.2 Astra and Sol are here! Power approaching the mighty Fable 5.2 at a fraction of the cost. The capabilities and use cases of OpenAI's two new models
- OrcaRouterRead ↗GPT-6.2 Astra Pro vs GPT-6.1 Sol: 4x the Price, 7 Points
Details
Tech Times reports that OpenAI’s system card says five research-engineering evaluations informed the High rating. The tests run in two stages: meeting an indicative cutoff on either first-stage test triggers the second stage. The card does not provide numeric cutoffs for the first-stage tests, so readers cannot determine from its published figures whether Astra met them.
On the second-stage tests, Astra scored 42.87% on NanoGPT against an 80% cutoff and 74.79% on PostTrainBench Lite against a 90% cutoff. MLE-Bench Revised was the only second-stage test whose cutoff Astra cleared; Sol cleared it as well. The card does not explain how OpenAI weighed the five evaluations in making its rating.
OpenAI describes the cutoffs as indicative and the High rating as an early milestone for preparing safeguards against AI self-improvement. The card also cites outside evaluations with mixed findings: Epoch AI found gains on several benchmarks but no meaningful improvement on its automation-report tasks, while Vals AI scored Astra 29.01% on a modified RSI Index, below the 50% human-level mark.
Written from coverage as of
How this story developed
- OpenAI releases GPT-6.2 Astra and Sol for ChatGPT, Codex and API users
- OpenAI rates Astra High for self-improvement despite missing two test cutoffsthis story
Images
- CNET
- note
- orcarouter.ai
- R&D World