Security Assessment of DeepSeek Harness with A.I.G: Evaluating Resistance to Indirect Prompt Injection

arXiv:2608.163932026-08-17

Researchers ran 14,560 real tests to see if hidden instructions in files an AI agent reads can trick it into taking harmful actions

Researchers used a testing tool called AI-Infra-Guard (A.I.G) to probe DeepSeek Harness (DSH), an open-source agent framework, for 'indirect prompt injection' - attacks hidden inside content the AI reads rather than typed by the user. They ran 14,560 real executions combining 16 content channels (web pages, emails, files, skills), two delivery formats (plain text vs. real files), 35 attack goals, and 12 attack techniques. In some specific conditions, the agent attempted the attacker's hidden instruction up to 25.5% of the time.

METAL MEDIA explanatory visual

Researchers ran 14,560 real tests to see if hidden instructions in files an AI agent reads can trick it into taking harmful actions

01The study measured whether DSH would carry out sensitive actions like sending email, running commands, or transferring money when a malicious instruction was hidden inside content it reads, such as web pages, emails, files, or reusable 'skills'
02A.I.G injected tainted content into DSH's actual TypeScript runtime, then captured detailed execution traces judged by two evaluators: a rule-based judge (J_R) and a semantic LLM-based judge (J_L)
03Overall full-success rate was 5.6% under J_R, but specific combinations were much riskier: 17.0% for text-based 'fake-completion' attacks, 25.5% for hidden Unicode characters delivered as files, and 16.0% for the skills channel in file form
04The same attack type behaved very differently depending on delivery format - hidden Unicode scored 0% success as plain text but 25.5% as an actual file, showing text-only testing can miss real risks
05The authors argue the problem isn't one flawed model but the entire pipeline - from ingesting content, to parsing it, to exposing it to the model, to planning, to authorizing tool calls - and recommend preserving content's origin information and requiring independent approval for sensitive actions

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

The study measured whether DSH would carry out sensitive actions like sending email, running commands, or transferring money when a malicious instruction was hidden inside content it reads, such as web pages, emails, files, or reusable 'skills'
A.I.G injected tainted content into DSH's actual TypeScript runtime, then captured detailed execution traces judged by two evaluators: a rule-based judge (J_R) and a semantic LLM-based judge (J_L)
Overall full-success rate was 5.6% under J_R, but specific combinations were much riskier: 17.0% for text-based 'fake-completion' attacks, 25.5% for hidden Unicode characters delivered as files, and 16.0% for the skills channel in file form
The same attack type behaved very differently depending on delivery format - hidden Unicode scored 0% success as plain text but 25.5% as an actual file, showing text-only testing can miss real risks
The authors argue the problem isn't one flawed model but the entire pipeline - from ingesting content, to parsing it, to exposing it to the model, to planning, to authorizing tool calls - and recommend preserving content's origin information and requiring independent approval for sensitive actions

Figure 2: Outcome criteria and evaluators. 𝒥R uses explicit trace evidence. 𝒥L reads the complete trace and the stated task-specific criterion. A sink call can support either full success or partial compliance.

Table 1: Assessment components used in the DSH study.
Component	Role in this study	Examples
Attack method	Expresses the attacker objective	fake completion, obfuscation
Indirect channel	Embeds the objective in content	webpage, skill, PDF metadata
DSH adapter	Executes the test in the target runtime	agent loop, tool registry, session log
Trace	Records messages, calls, results, and taint evidence	source-to-sink path
Judge	Assigns an outcome to the trace	𝒥R, 𝒥L

Figure 3: A.I.G runtime adapter for DSH. A.I.G prepares a tainted artifact and controlled fixture files. The real DSH TypeScript runtime handles the agent turn. The adapter converts DSH session events into a trace for 𝒥R and 𝒥L.

Table 2: Configuration of the assessed DSH deployment.
Item	Configuration
Runtime	DeepSeek Harness TypeScript runtime
Base cases	1,120 cases over 16 channels, 2 modes, and 35 objectives
Attack suite	13 methods: naive plus 12 attack methods
Agent executions	14,560 controlled runs
Carrier modes	560 text-mode and 560 file-mode base cases
Objectives	32 sink-required and 3 output-only tasks
Fixtures	6 source tools and 8 tracked simulated sinks
Model backend	deepseek-v4-flash [5] through a local proxy
Evaluators	Online 𝒥R and offline 𝒥L

Figure 4: DSH source-to-sink path. A tool result can become model-visible context, after which the model may select another tool. DSH also exposes pre-execution and deny-only guard hooks that a deployment can use to block risky calls.

Table 3: Outcome distribution over 14,560 DSH executions. Full and partial are mutually exclusive.
Outcome	𝒥R	𝒥L
Full success	819 (5.6%)	772 (5.3%)
Partial compliance	298 (2.0%)	1,060 (7.3%)
Broad influence (full + partial)	1,117 (7.6%)	1,832 (12.6%)
Failure / not reached	13,443 (92.4%)	12,719 (87.4%)
Judge error	0	9 (0.1%)

Figure 5: Dataset composition. The base dataset balances 16 channels, two carrier modes, and 35 payload objectives. The attack suite uses naive as an unmodified baseline and compares it with 12 attack methods.

Table 4: Outcomes by carrier mode. Full and partial values are percentages; sink calls are counts and overlap with the outcome categories.
	𝒥R	𝒥L
Mode	Runs	Full	Partial	Full	Partial
Text	7,280	5.1%	2.1%	5.2%	7.5%
File	7,280	6.2%	2.0%	5.4%	7.0%

Figure 6: Overall and selected attack-method results. The left panel shows full, partial, and non-success outcomes. The right panel compares selected attack methods. The vertical axis reports full-success rate (%).

Table 5: Attack success rates (%) by attack method and carrier mode. naive is the unmodified baseline.
Method	Rule Text	Rule File	LLM Text	LLM File
naive (baseline)	5.5%	5.9%	5.7%	5.5%
escape	8.6%	9.3%	9.3%	9.3%
context_ignoring	0.2%	1.8%	0.4%	1.6%
fake_completion	16.6%	16.1%	17.0%	15.5%
combined	1.2%	2.9%	1.2%	2.1%
payload_splitting	2.1%	4.6%	3.4%	4.5%
obfuscation	13.6%	13.6%	9.1%	8.8%
prefix_injection	3.8%	5.5%	5.0%	4.5%
format_confusion	5.0%	6.8%	5.0%	5.0%
context_flooding	2.7%	4.1%	1.8%	2.5%
cross_channel	3.0%	4.1%	5.0%	4.3%
important_instructions	2.9%	3.8%	3.9%	4.3%
stealth_instruction	0.9%	1.8%	1.1%	2.1%

Table 6: 𝒥R full-success rates (%) by channel. Each channel-mode cell contains 455 runs.
Channel	Text	File	Channel	Text	File
calendar_event	7.7%	4.2%	chat_message	9.5%	1.3%
code_comment	3.5%	3.5%	config_file	4.6%	2.4%
document	5.5%	7.3%	email_headers	3.3%	3.3%
log_file	2.4%	2.4%	markdown	1.8%	2.9%
pdf_metadata	6.4%	3.3%	skills	14.3%	16.0%
spreadsheet	3.5%	5.3%	structured_data	4.0%	4.2%
transaction_record	2.0%	4.0%	translation	7.9%	8.8%
unicode_hidden	0.0%	25.5%	webpage	5.1%	4.4%

Table 7: Key local artifacts used for the DSH case study.
Artifact	Role
A.I.G assessment components	Payload construction, taint routing, and trace normalization
𝒥R / 𝒥L	Deterministic and semantic outcome evaluation
full_channel_mode dataset	1,120-case text/file channel matrix
experiment runner	Matrix orchestration and online 𝒥R evaluation
DSH runtime adapter	Python-to-TypeScript harness bridge and trace mapping
DSH driver	Real DSH runtime initialization and session-event export
controlled test plugin	Source fixtures and tracked simulated sinks
final trace corpus	One normalized trace for each planned agent run
offline judge corpus	Trace-level 𝒥L verdicts

Why it matters

This matters for anyone building AI agents that read external documents, emails, or web content, since it shows concrete failure rates rather than abstract risk. It demonstrates that safety depends not just on the underlying model but on how the surrounding agent system handles untrusted content and authorizes actions.

Terms in this paper

indirect prompt injection · an attack where malicious instructions are hidden in content the AI reads (not typed by the user) to manipulate its behavior
source-to-sink · the path from where attacker-controlled content enters (source) to where a sensitive action is actually executed (sink)
tainted content · content that has been altered to contain a hidden malicious instruction
J_R / J_L judges · two ways of scoring whether an attack succeeded: one using fixed rules, the other using an AI to judge meaning
skills channel · reusable instruction sets registered for the agent to use, which can themselves become an attack vector

Original abstract (English)

We assess indirect prompt injection in DeepSeek Harness (DSH), using AI-Infra-Guard (A.I.G) to construct tests, deliver controlled taint, execute DSH, collect traces, and judge outcomes. The study covers 14,560 controlled executions over 16 indirect-content channels, text and file carrier modes, 35 payload objectives, one unmodified baseline, and 12 attack methods. The experiment preserves DSH's agent loop, tool registry, model adapter, and session-event path; source tools and sensitive sinks are local fixtures, so attempted actions are recorded without external side effects. We evaluate each trace with a deterministic rule-based judge, (RuleJudge), and a semantic LLM-based judge, (LLMJudge). The strongest observed attack success rates are 17.0% under for fake-completion attack in text mode, 25.5% under for hidden Unicode in file mode, and 16.0% under for the skills channel in file mode. also assigns partial compliance more often than (7.3% versus 2.0%). We relate these results to DSH's treatment of tool results, additional contexts, and tool-call policy hooks, then identify controls that should sit between untrusted content and sensitive actions. Our code is available at https://github.com/Tencent/AI-Infra-Guard/tree/main/Research/deepseek-harness-security-assessment .

Authors · Zonghao Ying

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Zonghao Ying et al., arXiv:2608.16393, CC BY 4.0