K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

Adversarial Review: Structured Disagreement for Grounded Agentic Code Review

arXiv:2608.181672026-08-20

AI代码审查:与其堆更多智能体,不如让一个审查者和一个批评者互相较真

研究者发现,给AI编程系统堆更多智能体带来的收益会逐渐减少,于是他们设计了一种精简协议Adversarial Review(AR),只用一个审查者智能体和一个批评者智能体互相争论,主编程智能体等两者意见稳定后才修改代码。AR在代码生成基准上击败了用五个智能体的对手方法,但在真实拉取请求审查基准上,两个智能体一开始过于容易达成没有依据的一致,后来通过强制批评者引用具体代码证据来反驳才解决了这个问题,改进后的版本在该基准上也拿到最高分,在真实修复Bug的大型基准上同样优于基线,不过计算成本更高。

METAL MEDIA 解读图

AI代码审查:与其堆更多智能体,不如让一个审查者和一个批评者互相较真

  1. 01在LiveCodeBench编程题基准上,只用一个主智能体加一个审查者、一个批评者的AR通过率达到87%,高于使用五个智能体（三个审查者加一个元审查者）的对手方法MARS的85%
  2. 02在真实GitHub拉取请求审查基准SWE-PRBench上,朴素版AR的得分反而是所测方法中最低的（F1为0.457）,原因是审查者和批评者常常在缺乏证据的情况下就相互认同,作者称之为“虚假共识”
  3. 03仅修改批评者的提示词——把简单的“同意/不同意”改成三种选项（同意、引用具体代码证据反驳、提出无证据支持的疑虑）——就把AR的F1提升到0.533,成为该测试中的最高分
  4. 04在要求修复500个真实GitHub问题的SWE-bench Verified基准上,AR的解决率达到75.2%,高于无协作基线的71.6%和MARS的72.6%,但AR消耗的计算token约为基线的4.5倍
  5. 05该设计把代码修改和审查讨论严格分开:审查者和批评者之间只交换审查文字,主智能体只有在两者就审查结果达成稳定、有证据支撑的一致后才会真正修改代码
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 在LiveCodeBench编程题基准上,只用一个主智能体加一个审查者、一个批评者的AR通过率达到87%,高于使用五个智能体（三个审查者加一个元审查者）的对手方法MARS的85%
  2. 在真实GitHub拉取请求审查基准SWE-PRBench上,朴素版AR的得分反而是所测方法中最低的（F1为0.457）,原因是审查者和批评者常常在缺乏证据的情况下就相互认同,作者称之为“虚假共识”
  3. 仅修改批评者的提示词——把简单的“同意/不同意”改成三种选项（同意、引用具体代码证据反驳、提出无证据支持的疑虑）——就把AR的F1提升到0.533,成为该测试中的最高分
  4. 在要求修复500个真实GitHub问题的SWE-bench Verified基准上,AR的解决率达到75.2%,高于无协作基线的71.6%和MARS的72.6%,但AR消耗的计算token约为基线的4.5倍
  5. 该设计把代码修改和审查讨论严格分开:审查者和批评者之间只交换审查文字,主智能体只有在两者就审查结果达成稳定、有证据支撑的一致后才会真正修改代码
Table 1: LCB results. The leader is bold. The first four methods cluster at the same pass-rate. MARS breaks out of the cluster. AR scores highest with the fewest agents among the methods that broke out.
Methodpass / 105pass-on-hard / 57# agents
Zero-shot77%35/57 (61%)1
Self-Refine77%35/57 (61%)1
Single-reviewer77%36/57 (63%)2
Two-reviewers75%34/57 (60%)3
MARS82%39/57 (68%)5
AR87%43/57 (75%)3
Table 2: SWE-PRBench review-relevant subset. The leader is bold. “AR” and “AR with text constraint” share the same R+C structure; only the prompts change. The number of agents and the way they connect did not change. The prompt iteration is described in Section 4.4.
MethodF1N
AR with text constraint0.533100
Two-reviewers0.503100
MARS0.501100
Single-reviewer0.495100
AR0.457100
Table 3: SWE-bench Verified pass-rate of three methods: Zero-shot, MARS, and AR.
Methodpass-rate (%)N
AR75.2%500
Zero-shot71.6%500
MARS72.6%500

为什么重要

随着企业越来越多地部署AI智能体自动编写和审查代码,这项研究表明单纯增加智能体数量并非答案,一种小而结构化的“对抗式”分歧机制反而更有效也更省成本。它也揭示了依赖多智能体AI审查的一个具体风险:智能体之间可能互相盲目附和彼此的错误,因此系统必须被明确设计成强制基于证据的反驳,而不是轻易达成一致。

本文术语

  • LLM智能体 · 基于大语言模型构建、能够规划并执行任务、有时会调用工具的程序
  • 子智能体 · 主智能体为完成特定子任务而调用的辅助AI智能体,使用方式类似工具
  • 多智能体辩论(MAD) · 多个AI智能体互相交换论点以完善答案的机制
  • 通过率 · AI正确解决任务或通过自动化测试的比例
  • F1分数 · 综合精确率和召回率的评价指标,这里用来衡量AI审查评论与真实人类评审意见的匹配程度

无法转载的图表

  • Figure 1: Workflow of Adversarial Review (AR). The main agent first produces artifact version N (code or plan). The protocol then enters an inner loop in which the artifact is frozen: reviewer R generates Reviewk, critic C evaluates and may revise that review, and R responds until a consistent review is reached. If the review converges on the first pass and identifies no flaws, the artifact is accepted immediately. Otherwise, the main agent edits the artifact to produce version N+1, updates the change log, and the process repeats. The key separation is that the inner loop exchanges review text only, while artifact edits occur only in the outer loop.
在原文中查看图表 →

论文原文摘要（英文）

Early multi-agent LLM systems often used role-separated teams, yet scaling agent count yields diminishing returns on repository-level coding tasks. Recent alternatives treat agents as passive tools (subagents), yet this removes the benefits of agent interaction entirely. We study whether a subagent paradigm can support a middle ground: minimal agentic cooperation without the overhead of large multi-agent teams. We introduce Adversarial Review (AR), a minimal cooperative code-review protocol in which a main coding agent works with a reviewer and a critic agent. The reviewer evaluates code, while the critic audits the review through structured disagreement before the main agent edits. On LiveCodeBench, AR achieves the highest pass rate among tested methods, outperforming a five-agent baseline while using only three agents. On SWE-PRBench, naive AR exposes a false-consensus failure mode, where agents converge on agreement without sufficient evidence, but a single prompt iteration that adds disagreement explicitly achieves the highest F1 among tested methods. On SWE-bench Verified, AR also shows improvements over the baselines on repository-level coding tasks. Together, AR demonstrates that cooperative code review does not require many agents or complex communication structures: it requires that disagreement be minimal, structured, and evidence-grounded.

作者 · Eric S. Qiu, Joyce Gill

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道