K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

arXiv:2608.051082026-08-04

不靠强化学习重新训练,靠写攻击笔记来持续变强的AI代理红队系统

PIMiner是一个自动寻找“提示注入”攻击的系统,这类攻击会把恶意指令偷偷藏进LLM代理读取的内容里。它不像强化学习方法那样为每个新目标重新训练攻击模型,而是把攻击经验积累成可复用的Markdown策略文件,无需重新训练就能迁移到未见过的目标模型上。在IPIArena和AgentDojo基准上,它达到了与最先进的强化学习方法相当的攻击成功率。

METAL MEDIA 解读图

PIMiner的四阶段攻击与学习流程

证据状态已报告实测结果

  1. 策略路由器针对每个新样本,只从不断增长的策略库中挑选最相关的前K个策略文件交给攻击者,而非加载全部内容,以此降低成本
  2. 迭代攻击模块结合长期策略、同一数据集内此前样本的经验、当前样本此前尝试的记录三种记忆,对注入提示进行最多10次迭代优化
  3. 经验整理器每轮结束后按攻击机制分析成功与失败的攻击案例,据此完善已有策略或新建策略文件,更新策略库
  4. 策略库记录目标模型适用范围、可用任务范围、注入模板、成功示例和已知失败条件的一组Markdown文件,无需重新训练即可迁移到未见过的模型
  5. 结果在IPIArena、AgentDojo和InjecAgent上,无需重新训练攻击者就达到了与最先进强化学习方法相当的攻击成功率
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 提示注入攻击是把恶意指令藏在网页、搜索结果或工具返回结果等AI代理会读取的不可信文本中,诱使代理执行用户本意之外的操作。红队测试,即主动尝试攻击以发现漏洞,是构建防御手段的关键前提。
  2. 现有方法分两派:基于强化学习的攻击者效果好,但训练需要数万次查询,而且很难迁移到新的目标模型;基于搜索的攻击者(如PAIR、TAP)每个样本都从零开始尝试,无法复用以往经验,因此效果较差。
  3. PIMiner整合了策略路由器、迭代攻击模块和经验整理器三个部分。成功的攻击模式会被保存进一个策略库(一组Markdown文件),供后续攻击复用。记忆分三层:跨数据集和模型的长期策略库、同一数据集内的中期经验、当前样本此前尝试记录的短期记忆。
  4. 为避免每次都把不断增长的整个策略库塞进攻击模型的上下文,路由器只挑选最相关的前K个策略文件传给攻击者,从而降低成本。实验中路由器将攻击者的输入长度缩短了43%到61%,同时攻击成功率基本保持不变甚至略有提升。
  5. 在IPIArena和AgentDojo上针对GPT、Claude、Gemini、DeepSeek系列共9个前沿大模型进行测试,PIMiner对Gemini-2.5-Pro在IPIArena上达到76.2%的攻击成功率,在AgentDojo上达到86.7%;在InjecAgent基准上,它与最先进的强化学习方法(RL-Hammer、PISmith)打成平手,成功率均为1.0。
Figure 1: PIMiner Pipeline. Given a dataset–model pair, PIMiner routes each sample to the most relevant strategies, performs iterative attack optimization, and then digests the resulting experiences to update its strategy library. During iterative attack optimization, an intra-dataset memory is maintained to facilitate knowledge sharing across samples within the same dataset.
Figure 1: PIMiner Pipeline. Given a dataset–model pair, PIMiner routes each sample to the most relevant strategies, performs iterative attack optimization, and then digests the resulting experiences to update its strategy library. During iterative attack optimization, an intra-dataset memory is maintained to facilitate knowledge sharing across samples within the same dataset.
Table 1: ASRs on InjecAgent. RL-based attacks and our PIMiner report ASR@10. Bold: best per row.
StaticSearch-BasedRL-BasedAgent-Based
ModelDirectEnhancedTAPPAIRStrategyGRPORL-Ham.PISmithPIMiner
GPT-4o-mini0.020.030.400.240.380.601.01.01.0
GPT-4.1-nano0.010.020.540.320.650.751.01.01.0
GPT-5-nano0.000.000.080.010.180.240.961.01.0
Figure 2: Evaluation of PIMiner on 9 frontier LLMs on IPIArena [23] and AgentDojo [5]. We report ASR@​1, ASR@​5, and ASR@​10.
Figure 2: Evaluation of PIMiner on 9 frontier LLMs on IPIArena [23] and AgentDojo [5]. We report ASR@​1, ASR@​5, and ASR@​10.
Table 2: ASRs on AgentDojo. PISmith and our PIMiner report ASR@10. Bold: best per row.
ModelDirectIgnore Prev.Sys. MsgInjecagentTool Know.Imp. Instr.PISmithPIMiner
GPT-4o-mini0.030.060.030.040.120.230.780.73
GPT-4.1-nano0.040.130.040.050.200.200.810.63
GPT-5-nano0.020.000.010.000.010.010.380.53
((b))
((b))
Table 3: Composition of the PIMiner training and test sets by domain. These datasets are randomly sampled.
BenchmarkDomain# Train# Test
AgentDojoBanking14
Slack33
Travel66
Workspace1017
Total2030
IPIArenaTool711
Browser35
Coding105
Total2021
Figure 4: The strategy library learned by PIMiner improves attack performance for a wide range of attacker LLMs.
Figure 4: The strategy library learned by PIMiner improves attack performance for a wide range of attacker LLMs.
Table 4: Structure of a PIMiner strategy file, which is specified in _TEMPLATE.md. The last column marks the primary consumer: Router, Attacker, or Digester.
SectionDescriptionConsumer
Title & introOne-line file description plus a 2–4 sentence summary of the attack mechanism, the target task domains it lands on, and how it differs from sibling strategies.A/D
Target-LLM scopePer-target verdicts: confirmed-effective, likely-effective, and not-shown-to-transfer (with run-dir citations), plus a one-line “use when” predicate.R/A/D
Task scopeWhich suites, injection-goal verbs (send_*, schedule_*, …), placeholder surface, and prerequisite conditions the strategy is known to apply to.R/A/D
Mechanism distinction2–5 sentences contrasting the core structural move against each named sibling strategy, so the router can disambiguate superficially similar options.A/D
Strategy templateThe abstract recipe: numbered, named structural moves (not surface text) that every in-context example instantiates.A/D
In-context examplesOne block per confirmed hit: verbatim user task, injection goal, full polluted placeholder context, winning injection text, resulting tool call, and why it worked.R/A/D
Fingerprint tableA compact, one-row-per-example view of the in-context examplesA/D
Failure conditionsDocumented structural failure conditions (not “model refused”), and where possible cite a run-dir / sample that demonstrated the failure, used to steer miss-pattern analysis.A/D
Iterative-attack init notesSeed guidance for the inner attacker: iter-0 framing, iter-to-iter escalation, and failure-mode pivots.A/D
Figure 5: Comparison of PIMiner with and without the router, in terms of the attacker agent’s average input length (a) and the resulting ASR (b).
Figure 5: Comparison of PIMiner with and without the router, in terms of the attacker agent’s average input length (a) and the resulting ASR (b).
Table 5: Prompt-injection strategies learned by PIMiner.
StrategyDescription
Fabricated Procedure GateImpersonates an authoritative website or system protocol that presents the malicious tool call as a mandatory prerequisite before the agent can proceed.
Forged Chat TurnInjects role-delimiter tokens to terminate the tool-output turn and fabricate a new user turn, causing the target to interpret the attacker’s directive as a legitimate user instruction.
Entity Data PoisoningEmbeds the attacker’s payload within a field explicitly requested by the user, making the malicious string load-bearing and encouraging provenance-aware models to reproduce it as part of an otherwise correct response.
Authenticated-Principal Voice ForgeFabricates a continuation message from an already authenticated principal (or from the user), allowing the malicious directive to inherit an authority chain the target already trusts.
Appended Output DirectiveAppends a forged “required answer format” instruction that coerces the model into emitting an attacker-chosen sentence in its final response.
Authority-Shed DocstringDisguises the malicious payload as a structural element of the code (e.g., a docstring or helper definition) rather than an instruction, encouraging the model to treat it as inert program content instead of executable guidance.
False-History ForgeFabricates an entire interaction history (user → assistant tool call → tool failure → user retry), framing the malicious tool invocation as a continuation of a previously authorized workflow rather than a new authorization request.
((b))
((b))
Table 6: An estimate of the training-phase cost. With access to a Claude Code subscription plan, the out-of-pocket cost for model API usage is approximately $20. In (M) and Out (M) denote the numbers of input and output tokens, respectively, measured in millions of tokens. For the attacker agent, In (M) is estimated based on the total number of input tokens, approximately 80% of which are cache hits.
ComponentModelIn (M)Out (M)Est. Cost ($)Use Claude Code
AttackerOpus 4.7∼78.0∼1.60∼115Yes
RouterOpus 4.7∼0.8∼0.08∼7Yes
DigesterOpus 4.7∼0.48∼0.12∼5Yes
Target LLMmixed∼7.9∼0.65∼20No
Figure 6: Distribution of in-context examples from the seven distinct attack strategies across different target LLMs.
Figure 6: Distribution of in-context examples from the seven distinct attack strategies across different target LLMs.

研究结果

  • 在IPIArena上,PIMiner对Gemini-2.5-Pro达到76.2%的ASR@10,对GPT-5.1达到61.9%,对Claude-Sonnet-4.5达到42.9%。
  • 在AgentDojo上,对Gemini-2.5-Pro达到86.7%,对GPT-5.1达到53.3%,对Claude-Sonnet-4.5达到40.0%;而Claude-Opus-4.5的抗攻击能力远超其他模型,ASR仅为4.8%(IPIArena)和3.3%(AgentDojo)。
  • 在InjecAgent基准上,PIMiner对GPT-4o-mini、GPT-4.1-nano、GPT-5-nano三个模型均达到ASR为1.0,与最好的强化学习方法(RL-Hammer、PISmith)持平,而基于搜索的PAIR和TAP在GPT-5-nano上分别只有1%和8%。
  • 在AgentDojo上,PIMiner在GPT-5-nano上优于PISmith(53% vs 38%),在GPT-4o-mini上相当(73% vs 78%),在GPT-4.1-nano上略逊(63% vs 81%)——但PISmith需要为每个目标模型单独训练攻击者,而PIMiner是直接应用于训练阶段未见过的模型。
  • 去掉路由器后,攻击者的输入长度在Claude-Haiku-4.5上增加了43%,在Claude-Sonnet-4.6上增加了61%,但攻击成功率并未因此明显提升(加入路由器后ASR仅下降1.2%或反而上升7.5%)。同时使用长期策略库和数据集内记忆,相比普通迭代攻击者,平均ASR提升了17.8到19.8个百分点。

可应用场景

  • 在部署AI代理之前,以较低成本排查提示注入方面的安全漏洞
  • 生成高质量的攻击数据,用于训练防护模型或改进主干大模型的对齐
  • 持续性的红队测试工作流,面对新发布的目标模型时直接复用已有策略库,无需重新训练攻击者
  • 每个样本只能进行有限次查询(例如10次)的黑盒评估场景

局限与待验证事项

  • 面对防御能力特别强的模型(如Claude-Opus-4.5)时成功率仍然很低(4.8%和3.3%),说明该方法并非对所有目标都同样有效
  • 训练阶段假设可以观察目标代理的中间输出(灰盒访问),这一假设主要适用于开源或公开中间过程的代理,而非完全封闭的系统
  • 策略库中部分策略(如Authority Shed Docstring、False History Forge)仅有一个示例支撑,其在更多模型上的泛化能力尚不清楚
  • 评测仅限于IPIArena、AgentDojo、InjecAgent这几个基准以及训练中使用的八组数据集-模型组合,在完全不同类型的代理或任务领域上的表现尚未验证
  • 论文中约20美元的训练成本估算基于拥有Claude Code订阅套餐的前提,若纯按API调用计费,成本结构可能明显不同

为什么重要

各公司在部署AI代理之前都需要排查提示注入漏洞,并收集数据来训练更强的防御模型,但现有方法成本高昂且难以迁移到新模型。像PIMiner这样把经验沉淀为可复用知识而非每次重新训练的方法,能让这类安全测试变得更便宜、更容易规模化开展。

本文术语

  • 提示注入 · 把恶意指令藏进AI代理会读取的文本(网页、工具返回结果等)中,以此操纵代理行为的攻击方式
  • 红队测试 · 主动扮演攻击者去测试系统,以便在真正的攻击者之前发现漏洞
  • 基于强化学习的攻击 · 通过奖励反馈反复训练攻击模型,使其生成越来越有效的攻击
  • 攻击成功率(ASR) · 尝试的攻击中实际成功的比例;ASR@N表示在N次尝试内只要有一次成功就算成功
  • 策略库 · 记录成功攻击模式、适用范围和示例的一组Markdown文件,供后续攻击复用

论文原文摘要（英文）

Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting training data to improve defenses. Existing state-of-the-art prompt injection red-teaming methods primarily rely on reinforcement learning (RL), producing attacker models that often generalize poorly to new target LLMs. In this work, we develop PIMiner, an agentic system for prompt injection red-teaming. During training, PI

作者 · Yanting Wang

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Yanting Wang et al., arXiv:2608.05108, CC BY 4.0