Comparison

AI Debate with Human Judges: Task and Comparator Limits

Debate uses opposing AI advocates before a judge; consultancy uses one AI advisor. Rahman and colleagues’ NeurIPS 2025 paper (first submitted 2025-06-02; v2 revision 2025-10-29) reports accuracy of people’s final judgments after debate versus consultancy: 70.1% versus 60.0% on selected COVID-19 claims and 72.1% versus 68.3% on climate claims. COVID consultants were randomly assigned positions, 50% correct; climate consultants chose positions, 92.5% correct. Judges lacked underlying sources. Carro and colleagues’ preprint v1 (2026-10-20) reports 41.30% to 57.61% accuracy when retesting the debated claim. Its analysis supported that increase; consultancy showed no clear change. Testing a different related claim after debate was inconclusive, with unmatched before/after items. Its author-written claims lacked expert verification. Selected nonexperts judged bounded facts; neither study tested communities validating or redirecting new AI-generated discoveries.

0

1

Updated 2026-11-10

Contributors are:

Who are from:

Tags

Scientific Oversight of AI-Assisted Research

1Cademy