Agent Against Agent: An Agentic System for Automatic Prompt Injection Red Teaming

arXiv:2608.051082026-08-04

A red-team system that keeps getting better at tricking AI agents by writing down what worked, instead of retraining a model

PIMiner automatically searches for prompt injection attacks that sneak malicious instructions into LLM agents. Instead of retraining an attacker model with reinforcement learning for every new target, it accumulates attack know-how into reusable markdown strategy files that transfer to unseen models without retraining. On IPIArena and AgentDojo it matched the attack success rates of state-of-the-art RL-based methods.

METAL MEDIA explanatory visual

PIMiner's four-stage attack-and-learn pipeline

Evidence statusMeasured results reported

Strategy RouterFor each new sample, selects only the top-K most relevant strategy files from the growing library instead of loading everything, cutting cost
Iterative Attack ModuleRefines the injected prompt up to 10 times per sample, drawing on long-term strategies, dataset-level experience, and the sample's own prior attempts
Experience DigesterAnalyzes successful and failed attacks by underlying mechanism after each run, refining existing strategies or creating new ones to update the library
Strategy LibraryMarkdown files recording target-model scope, applicable tasks, injection templates, examples, and known failure conditions — reusable on unseen models without retraining
ResultMatched state-of-the-art RL-based attack success rates on IPIArena, AgentDojo, and InjecAgent without any attacker retraining

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

Prompt injection hides malicious instructions inside untrusted text an AI agent reads, such as webpages, search results, or tool outputs, tricking the agent into doing something other than what the user asked. Red-teaming, deliberately probing for these vulnerabilities, is essential for building defenses.
Prior approaches split into two camps: RL-based attackers get strong results but need tens of thousands of training queries and transfer poorly to new target models; search-based attackers (PAIR, TAP) restart from scratch for every sample and never reuse past experience, so they underperform.
PIMiner combines a strategy router, an iterative attack module, and an experience digester. Successful attack patterns are saved into a strategy library of markdown files that get reused on future attacks. Memory operates at three levels: a long-term library shared across datasets and models, a mid-term memory of experience within the current dataset, and a short-term record of prior attempts on the current sample.
Rather than loading the entire, ever-growing strategy library into the attacker's context every time, a router agent selects only the top-K most relevant strategy files, cutting cost. In experiments the router shortened the attacker's input by 43-61% while keeping (or even slightly improving) attack success rates.
Tested against 9 frontier LLMs from the GPT, Claude, Gemini, and DeepSeek families on IPIArena and AgentDojo, PIMiner reached 76.2% attack success on Gemini-2.5-Pro on IPIArena and 86.7% on AgentDojo, and on InjecAgent it matched state-of-the-art RL-based methods (RL-Hammer, PISmith) with a perfect success rate of 1.0.

Figure 1: PIMiner Pipeline. Given a dataset–model pair, PIMiner routes each sample to the most relevant strategies, performs iterative attack optimization, and then digests the resulting experiences to update its strategy library. During iterative attack optimization, an intra-dataset memory is maintained to facilitate knowledge sharing across samples within the same dataset.

Table 1: ASRs on InjecAgent. RL-based attacks and our PIMiner report ASR@10. Bold: best per row.
	Static	Search-Based	RL-Based	Agent-Based
Model	Direct	Enhanced	TAP	PAIR	Strategy	GRPO	RL-Ham.	PISmith	PIMiner
GPT-4o-mini	0.02	0.03	0.40	0.24	0.38	0.60	1.0	1.0	1.0
GPT-4.1-nano	0.01	0.02	0.54	0.32	0.65	0.75	1.0	1.0	1.0
GPT-5-nano	0.00	0.00	0.08	0.01	0.18	0.24	0.96	1.0	1.0

Figure 2: Evaluation of PIMiner on 9 frontier LLMs on IPIArena [23] and AgentDojo [5]. We report ASR@1, ASR@5, and ASR@10.

Table 2: ASRs on AgentDojo. PISmith and our PIMiner report ASR@10. Bold: best per row.
Model	Direct	Ignore Prev.	Sys. Msg	Injecagent	Tool Know.	Imp. Instr.	PISmith	PIMiner
GPT-4o-mini	0.03	0.06	0.03	0.04	0.12	0.23	0.78	0.73
GPT-4.1-nano	0.04	0.13	0.04	0.05	0.20	0.20	0.81	0.63
GPT-5-nano	0.02	0.00	0.01	0.00	0.01	0.01	0.38	0.53

Table 3: Composition of the PIMiner training and test sets by domain. These datasets are randomly sampled.
Benchmark	Domain	# Train	# Test
AgentDojo	Banking	1	4
Slack	3	3
Travel	6	6
Workspace	10	17
Total	20	30
IPIArena	Tool	7	11
Browser	3	5
Coding	10	5
Total	20	21

Figure 4: The strategy library learned by PIMiner improves attack performance for a wide range of attacker LLMs.

Table 4: Structure of a PIMiner strategy file, which is specified in _TEMPLATE.md. The last column marks the primary consumer: Router, Attacker, or Digester.
Section	Description	Consumer
Title & intro	One-line file description plus a 2–4 sentence summary of the attack mechanism, the target task domains it lands on, and how it differs from sibling strategies.	A/D
Target-LLM scope	Per-target verdicts: confirmed-effective, likely-effective, and not-shown-to-transfer (with run-dir citations), plus a one-line “use when” predicate.	R/A/D
Task scope	Which suites, injection-goal verbs (send_, schedule_, …), placeholder surface, and prerequisite conditions the strategy is known to apply to.	R/A/D
Mechanism distinction	2–5 sentences contrasting the core structural move against each named sibling strategy, so the router can disambiguate superficially similar options.	A/D
Strategy template	The abstract recipe: numbered, named structural moves (not surface text) that every in-context example instantiates.	A/D
In-context examples	One block per confirmed hit: verbatim user task, injection goal, full polluted placeholder context, winning injection text, resulting tool call, and why it worked.	R/A/D
Fingerprint table	A compact, one-row-per-example view of the in-context examples	A/D
Failure conditions	Documented structural failure conditions (not “model refused”), and where possible cite a run-dir / sample that demonstrated the failure, used to steer miss-pattern analysis.	A/D
Iterative-attack init notes	Seed guidance for the inner attacker: iter-0 framing, iter-to-iter escalation, and failure-mode pivots.	A/D

Figure 5: Comparison of PIMiner with and without the router, in terms of the attacker agent’s average input length (a) and the resulting ASR (b).

Table 5: Prompt-injection strategies learned by PIMiner.
Strategy	Description
Fabricated Procedure Gate	Impersonates an authoritative website or system protocol that presents the malicious tool call as a mandatory prerequisite before the agent can proceed.
Forged Chat Turn	Injects role-delimiter tokens to terminate the tool-output turn and fabricate a new user turn, causing the target to interpret the attacker’s directive as a legitimate user instruction.
Entity Data Poisoning	Embeds the attacker’s payload within a field explicitly requested by the user, making the malicious string load-bearing and encouraging provenance-aware models to reproduce it as part of an otherwise correct response.
Authenticated-Principal Voice Forge	Fabricates a continuation message from an already authenticated principal (or from the user), allowing the malicious directive to inherit an authority chain the target already trusts.
Appended Output Directive	Appends a forged “required answer format” instruction that coerces the model into emitting an attacker-chosen sentence in its final response.
Authority-Shed Docstring	Disguises the malicious payload as a structural element of the code (e.g., a docstring or helper definition) rather than an instruction, encouraging the model to treat it as inert program content instead of executable guidance.
False-History Forge	Fabricates an entire interaction history (user → assistant tool call → tool failure → user retry), framing the malicious tool invocation as a continuation of a previously authorized workflow rather than a new authorization request.

Table 6: An estimate of the training-phase cost. With access to a Claude Code subscription plan, the out-of-pocket cost for model API usage is approximately $20. In (M) and Out (M) denote the numbers of input and output tokens, respectively, measured in millions of tokens. For the attacker agent, In (M) is estimated based on the total number of input tokens, approximately 80% of which are cache hits.
Component	Model	In (M)	Out (M)	Est. Cost ($)	Use Claude Code
Attacker	Opus 4.7	∼78.0	∼1.60	∼115	Yes
Router	Opus 4.7	∼0.8	∼0.08	∼7	Yes
Digester	Opus 4.7	∼0.48	∼0.12	∼5	Yes
Target LLM	mixed	∼7.9	∼0.65	∼20	No

Figure 6: Distribution of in-context examples from the seven distinct attack strategies across different target LLMs.

Findings

On IPIArena, PIMiner reached 76.2% ASR@10 against Gemini-2.5-Pro, 61.9% against GPT-5.1, and 42.9% against Claude-Sonnet-4.5.
On AgentDojo, it reached 86.7% against Gemini-2.5-Pro, 53.3% against GPT-5.1, and 40.0% against Claude-Sonnet-4.5, while Claude-Opus-4.5 was by far the most resistant target at only 4.8% (IPIArena) and 3.3% (AgentDojo).
On InjecAgent, PIMiner achieved ASR 1.0 across GPT-4o-mini, GPT-4.1-nano, and GPT-5-nano, matching the best RL-based baselines (RL-Hammer, PISmith), while search-based PAIR and TAP only reached 1% and 8% respectively on GPT-5-nano.
On AgentDojo, PIMiner outperformed PISmith on GPT-5-nano (53% vs. 38%), was comparable on GPT-4o-mini (73% vs. 78%), and lagged on GPT-4.1-nano (63% vs. 81%) — but PISmith needs a separately trained attacker per target model, while PIMiner was applied directly to models unseen during training.
Removing the router increased the attacker's input length by 43% (Claude-Haiku-4.5) and 61% (Claude-Sonnet-4.6) without meaningfully improving ASR (ASR dropped 1.2% or rose 7.5% with the router). Combining both the long-term strategy library and intra-dataset memory improved average ASR over a plain iterative attacker by 17.8-19.8 percentage points.

Where it can be used

Low-cost pre-deployment security testing to find prompt injection weaknesses in AI agents
Generating high-quality attack data to train guardrail models or better-aligned backbone LLMs
Ongoing red-teaming operations where a reusable strategy library is applied to newly released target models without retraining an attacker
Black-box evaluation settings limited to a small number of queries per sample (e.g., 10) against a target agent

Limits and open work

The method still struggles against especially robust models like Claude-Opus-4.5, where success rates remain low (4.8% and 3.3%), so it does not work uniformly well across all targets.
Training assumes grey-box access, observing intermediate outputs of the target agent, which is realistic mainly for open-source or transparency-exposing agents, not fully closed systems.
Some learned strategies (e.g., Authority Shed Docstring, False History Forge) are backed by only a single example, so how well they generalize beyond the models they were discovered on is unclear.
Evaluation is limited to specific benchmarks (IPIArena, AgentDojo, InjecAgent) and the eight dataset-model pairs used in training; performance on very different agent types or task domains is untested.
Cost estimates (about $20 for training) assume access to a Claude Code subscription plan; costs could differ substantially under a pure pay-per-API-call setup.

Why it matters

Companies need to check their AI agents for prompt injection weaknesses before shipping them and to collect data for training better defenses, but existing methods are expensive and don't transfer across models. A system that accumulates reusable knowledge instead of retraining from scratch each time could make this kind of security testing far cheaper and more scalable.

Terms in this paper

Prompt injection · An attack that hides malicious instructions in text an AI agent reads (webpages, tool outputs, etc.) to hijack its behavior
Red-teaming · Deliberately attacking a system to find its vulnerabilities before real attackers do
RL-based attack · Training an attacker model through reinforcement learning feedback so it generates increasingly effective attacks
ASR (Attack Success Rate) · The share of attempted attacks that actually succeed; ASR@N counts a sample as success if any of N tries works
Strategy library · A collection of markdown files recording successful attack patterns, their scope, and examples, kept for reuse on future attacks

Original abstract (English)

Prompt injection poses significant security risks to LLM agents. Efficient and effective red-teaming is therefore critical, both for evaluating these risks and for collecting training data to improve defenses. Existing state-of-the-art prompt injection red-teaming methods primarily rely on reinforcement learning (RL), producing attacker models that often generalize poorly to new target LLMs. In this work, we develop PIMiner, an agentic system for prompt injection red-teaming. During training, PI

Authors · Yanting Wang

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Yanting Wang et al., arXiv:2608.05108, CC BY 4.0