Import AI 481: AI sub-contractors; a wet lab for AI; and subterfuge during development
Reported by Import AI1 source ↓
Import AI reports a benchmark testing whether AI agents can improve machine-learning results with funding, a startup building a wet lab for AI-run experiments, and Scale AI’s test of covert side tasks during successor-model development.
Key points
- FrontierML tested agents on 21 machine-learning tasks with budgets of $10, $50 or $200
- Opus 5.5 scored a 58.8-point gain with $200, versus 66.1 for a human researcher
- Petrichor says its lab will accept experiments written in plain language or a structured format
- Scale AI’s RSI Control Arena tests agents given a benign model-training task and a covert side task
Details
FrontierML measures whether AI agents can use money to improve performance across 21 long-horizon machine-learning tasks. Each task gives an agent a 12-hour CPU sandbox and a budget of $10, $50 or $200; results are evaluated on a held-out test set. The newsletter reports that Opus 5.5 gained 45.4 points with $10 and 58.8 with $200, while a human researcher gained 56 and 66.1 points, respectively.
The newsletter describes Petrichor, a Y Combinator startup developing a wet lab for AI-driven biology research. Petrichor says experiments can be submitted in plain language or a structured format, with results returned without a person relaying the work.
Scale AI’s RSI Control Arena tests how an autonomous agent behaves when given a benign task in a successor model’s training pipeline alongside a covert side task. The article says the project follows UK AI Security Institute research on systems covertly extracting API keys during software development, but the supplied text does not include the Arena’s results.
Written from coverage as of