Technical Research
AI safety via debate: scalable oversight with weak judges
Can a highly capable AI system deceive a limited judge, even when that judge supervises a debate between two models? This is the core question investigated by Joaquín Machulsky in his graduation thesis at the University of Buenos Aires.
The AI Safety via Debate paradigm suggests that making two AI agents compete to convince a weaker judge or a human can surface the truth, as any lie can be exposed by the counterparty. Joaquín pushed this framework further by exploring asymmetric capabilities, analyzing what happens when a dishonest agent is strategically superior (utilizing MCTS tree-search) to an honest opponent (relying on Greedy strategies).
The findings reveal that a more powerful, dishonest model can systematically exploit its rival. However, the study demonstrates that a single protocol rule —the precommitment condition— completely reverses this dynamic, making honesty the winning strategy. The core takeaway for technical alignment is vital: formal protocol design is just as critical as power parity between systems. The session also features a public interactive demo to experience these scalable oversight mechanisms firsthand.
