How “reverse game theory” could keep AI agents in check
Reported by MIT Technology Review and Crypto Briefing2 sources ↓
Researchers are exploring “reverse game theory,” or mechanism design, to make rule-following the rational choice for AI agents. Proposed methods include cross-checking models and assigning agents to audit one another.
Key points
- One proposal rewards a model for predicting another model’s answers, making deliberate underperformance less useful
- Another pairs solver agents with auditor agents and penalizes both missed cheating and cheating that is caught
- Researchers question whether AI agents respond to incentives as mechanism design assumes
Sources
Details
MIT Technology Review reports that economist Stephen Morris of MIT and colleagues from Yale and Columbia outlined possible ways to discourage AI models from sandbagging safety tests in a preprint. One approach rewards a model based on how well its answers predict a second model’s responses; other proposals pit models against each other or pair models with opposing biases and reward agreement.
Princeton computer science professor Elad Hazan proposes having models take turns solving problems and auditing other models’ work. Solvers caught cheating would be punished, as would auditors who fail to detect cheating; disputed cases could go to a human expert. Hazan also suggests auditing the auditors to reduce the risk of collusion.
The proposals aim to shape agents’ incentives rather than depend on understanding their internal workings. The article notes a central uncertainty: mechanism design assumes agents respond rationally to incentives, and researchers question whether that assumption holds for AI models.
Written from coverage as of
Images
- MIT Technology Review