Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
For AI code review, one reviewer plus one critic beats piling on more agents
Researchers found that adding more AI agents to a coding team gives shrinking returns, so instead they built a lean protocol called Adversarial Review (AR) where just one reviewer agent and one critic agent argue over a code review before the main coding agent edits anything. AR beat a five-agent baseline on a coding benchmark, but on a real pull-request review benchmark the two agents initially just agreed with each other too easily, a problem the authors fixed by forcing the critic to always cite concrete code evidence when disagreeing. The fixed version then topped that benchmark too, and AR also outperformed baselines on a large real-world bug-fixing benchmark, though at higher computational cost.
METAL MEDIA explanatory visual
For AI code review, one reviewer plus one critic beats piling on more agents
- 01On LiveCodeBench (coding problems), AR — a main agent plus just one reviewer and one critic — reached an 87% pass rate versus 85% for MARS, a rival method that uses five agents (three reviewers plus a meta-reviewer)
- 02On SWE-PRBench, a benchmark of real GitHub pull-request reviews, plain AR actually scored lowest of the methods tested (F1 = 0.457) because the reviewer and critic tended to agree with each other without enough evidence, a failure the authors call 'false consensus'
- 03Changing only the critic's prompt — from a simple agree/disagree choice to three options (agree, disagree with cited code evidence, or disagree with an unverified concern) — raised AR's score to F1 = 0.533, the best in that test
- 04On SWE-bench Verified, which asks agents to fix 500 real GitHub issues, AR resolved 75.2% of tasks versus 71.6% for a no-collaboration baseline and 72.6% for MARS, though AR used about 4.5 times more computing tokens than the baseline
- 05The design keeps code edits separate from review discussion: the reviewer and critic can only exchange review text, and the main agent only edits the code after the two have reached a stable, evidence-backed agreement
What they did
- On LiveCodeBench (coding problems), AR — a main agent plus just one reviewer and one critic — reached an 87% pass rate versus 85% for MARS, a rival method that uses five agents (three reviewers plus a meta-reviewer)
- On SWE-PRBench, a benchmark of real GitHub pull-request reviews, plain AR actually scored lowest of the methods tested (F1 = 0.457) because the reviewer and critic tended to agree with each other without enough evidence, a failure the authors call 'false consensus'
- Changing only the critic's prompt — from a simple agree/disagree choice to three options (agree, disagree with cited code evidence, or disagree with an unverified concern) — raised AR's score to F1 = 0.533, the best in that test
- On SWE-bench Verified, which asks agents to fix 500 real GitHub issues, AR resolved 75.2% of tasks versus 71.6% for a no-collaboration baseline and 72.6% for MARS, though AR used about 4.5 times more computing tokens than the baseline
- The design keeps code edits separate from review discussion: the reviewer and critic can only exchange review text, and the main agent only edits the code after the two have reached a stable, evidence-backed agreement
| Method | pass / 105 | pass-on-hard / 57 | # agents |
|---|---|---|---|
| Zero-shot | 77% | 35/57 (61%) | 1 |
| Self-Refine | 77% | 35/57 (61%) | 1 |
| Single-reviewer | 77% | 36/57 (63%) | 2 |
| Two-reviewers | 75% | 34/57 (60%) | 3 |
| MARS | 82% | 39/57 (68%) | 5 |
| AR | 87% | 43/57 (75%) | 3 |
| Method | F1 | N |
|---|---|---|
| AR with text constraint | 0.533 | 100 |
| Two-reviewers | 0.503 | 100 |
| MARS | 0.501 | 100 |
| Single-reviewer | 0.495 | 100 |
| AR | 0.457 | 100 |
| Method | pass-rate (%) | N |
|---|---|---|
| AR | 75.2% | 500 |
| Zero-shot | 71.6% | 500 |
| MARS | 72.6% | 500 |
Why it matters
As companies increasingly deploy AI agents to write and review code automatically, this work shows that simply adding more agents to the team is not the answer — a small, tightly structured disagreement between two agents works better and cheaper. It also exposes a concrete risk for anyone relying on multi-agent AI review: agents can rubber-stamp each other's mistakes, so systems need to be explicitly designed to force evidence-based pushback rather than easy agreement.
Terms in this paper
- LLM agent · a program built on a large language model that plans and carries out tasks, sometimes using tools
- subagent · a helper AI agent that the main agent calls for a specific subtask, similar to using a tool
- multi-agent debate (MAD) · a setup where several AI agents exchange arguments back and forth to refine an answer
- pass rate · the percentage of tasks the AI solves correctly or that pass automated tests
- F1 score · a metric combining precision and recall, used here to measure how well AI review comments match real human reviewer comments
Figures we cannot republish
- Figure 1: Workflow of Adversarial Review (AR). The main agent first produces artifact version N (code or plan). The protocol then enters an inner loop in which the artifact is frozen: reviewer R generates Reviewk, critic C evaluates and may revise that review, and R responds until a consistent review is reached. If the review converges on the first pass and identifies no flaws, the artifact is accepted immediately. Otherwise, the main agent edits the artifact to produce version N+1, updates the change log, and the process repeats. The key separation is that the inner loop exchanges review text only, while artifact edits occur only in the outer loop.
Original abstract (English)
Early multi-agent LLM systems often used role-separated teams, yet scaling agent count yields diminishing returns on repository-level coding tasks. Recent alternatives treat agents as passive tools (subagents), yet this removes the benefits of agent interaction entirely. We study whether a subagent paradigm can support a middle ground: minimal agentic cooperation without the overhead of large multi-agent teams. We introduce Adversarial Review (AR), a minimal cooperative code-review protocol in which a main coding agent works with a reviewer and a critic agent. The reviewer evaluates code, while the critic audits the review through structured disagreement before the main agent edits. On LiveCodeBench, AR achieves the highest pass rate among tested methods, outperforming a five-agent baseline while using only three agents. On SWE-PRBench, naive AR exposes a false-consensus failure mode, where agents converge on agreement without sufficient evidence, but a single prompt iteration that adds disagreement explicitly achieves the highest F1 among tested methods. On SWE-bench Verified, AR also shows improvements over the baselines on repository-level coding tasks. Together, AR demonstrates that cooperative code review does not require many agents or complex communication structures: it requires that disagreement be minimal, structured, and evidence-grounded.
Read on arXivLatest papers
- SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?AI coding agents were tested on fixing real scientific software, and even the best one failed more than half the time
- FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papers
- PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM AgentsMaking customer-service AI agents follow the whole procedure, not just avoid one bad action
- EXIMO: VLM Guided Exploration of VLA PoliciesTeaching a robot new chores without human teleoperation, by letting a chatty AI supervise it
- EnvHarness: Awakening Static Worlds for Agent LearningInstead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknesses
- Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the ModelCompanies that rent AI instead of owning it can only do half of AI safety oversight
- Beyond Imitation: Filtering On-Policy Distillation by Reasoning ProgressA fix for AI models that get penalized by their teacher even when they're reasoning correctly
- PersonalBench: Measuring the Authorship Gap in LLM PersonalizationAI can be prompted to write 'like someone,' but its own voice never fully disappears