SPECS: Target Recall Is Not All-Objection Precision
Biswas and colleagues’ author-posted draft (arXiv v1, 2026-04-15) reports SPECS: 783 deliberate target edits to versions of 120 earlier research reports. An AI judged whether generated reviews—written critiques—caught each planted target. A system using several review steps detected 63.86% of targets versus 42.91% with a single review instruction. Recall means the share of targets detected; precision means the share of all raised objections—claims of a problem—that are justified. Two humans checked 35 edits: both judged 22 significant errors and 9 too minor, and disagreed on 4. That small audit concerns target labels, not all raised objections; neither result supplies precision for unedited reports.
0
1
Tags
Scientific Oversight of AI-Assisted Research
1Cademy