K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

Security Assessment of DeepSeek Harness with A.I.G: Evaluating Resistance to Indirect Prompt Injection

arXiv:2608.163932026-08-17

研究人员做了14560次真实测试,看AI智能体读取的文件里藏的指令能不能骗它去做危险操作

研究者用一款叫AI-Infra-Guard(A.I.G)的测试工具,对开源智能体框架DeepSeek Harness(DSH)进行了间接提示注入测试——也就是把恶意指令藏在AI要读取的内容里,而不是用户直接输入的。他们组合了16种内容渠道(网页、邮件、文件、技能库等)、两种投递方式(纯文本和真实文件)、35个攻击目标和12种攻击手法,总共跑了14560次真实执行。结果显示,在某些特定条件下,智能体尝试执行隐藏指令的比例最高达到25.5%。

METAL MEDIA 解读图

研究人员做了14560次真实测试,看AI智能体读取的文件里藏的指令能不能骗它去做危险操作

  1. 01研究测试的是:当网页、邮件、文件或可复用的'技能'模块里藏有恶意指令时,DSH智能体是否会因此发送邮件、执行命令、转账等敏感操作
  2. 02A.I.G把带毒内容注入DSH真实的TypeScript运行环境,记录完整执行轨迹,再用两种评判方式打分:基于规则的判定器(J_R)和基于语言模型语义理解的判定器(J_L)
  3. 03整体完全成功率在J_R下为5.6%,但特定组合风险高得多:文本形式的'伪完成(fake-completion)'攻击达17.0%,以文件形式隐藏的Unicode字符达25.5%,文件形式的技能渠道达16.0%
  4. 04同一种攻击手法用文本和用真实文件投递时结果差异巨大——隐藏Unicode以文本形式测试成功率为0%,换成文件形式却飙升到25.5%,说明只用文本测试会漏掉真实风险
  5. 05作者认为问题不在于某个模型本身的缺陷,而在于从内容摄取、解析、呈现给模型、规划到工具调用授权的整条链路,建议保留内容来源信息,并让敏感操作的授权独立于模型的判断
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 研究测试的是:当网页、邮件、文件或可复用的'技能'模块里藏有恶意指令时,DSH智能体是否会因此发送邮件、执行命令、转账等敏感操作
  2. A.I.G把带毒内容注入DSH真实的TypeScript运行环境,记录完整执行轨迹,再用两种评判方式打分:基于规则的判定器(J_R)和基于语言模型语义理解的判定器(J_L)
  3. 整体完全成功率在J_R下为5.6%,但特定组合风险高得多:文本形式的'伪完成(fake-completion)'攻击达17.0%,以文件形式隐藏的Unicode字符达25.5%,文件形式的技能渠道达16.0%
  4. 同一种攻击手法用文本和用真实文件投递时结果差异巨大——隐藏Unicode以文本形式测试成功率为0%,换成文件形式却飙升到25.5%,说明只用文本测试会漏掉真实风险
  5. 作者认为问题不在于某个模型本身的缺陷,而在于从内容摄取、解析、呈现给模型、规划到工具调用授权的整条链路,建议保留内容来源信息,并让敏感操作的授权独立于模型的判断
Figure 2: Outcome criteria and evaluators. 𝒥R uses explicit trace evidence. 𝒥L reads the complete trace and the stated task-specific criterion. A sink call can support either full success or partial compliance.
Figure 2: Outcome criteria and evaluators. 𝒥R uses explicit trace evidence. 𝒥L reads the complete trace and the stated task-specific criterion. A sink call can support either full success or partial compliance.
Table 1: Assessment components used in the DSH study.
ComponentRole in this studyExamples
Attack methodExpresses the attacker objectivefake completion, obfuscation
Indirect channelEmbeds the objective in contentwebpage, skill, PDF metadata
DSH adapterExecutes the test in the target runtimeagent loop, tool registry, session log
TraceRecords messages, calls, results, and taint evidencesource-to-sink path
JudgeAssigns an outcome to the trace𝒥R, 𝒥L
Figure 3: A.I.G runtime adapter for DSH. A.I.G prepares a tainted artifact and controlled fixture files. The real DSH TypeScript runtime handles the agent turn. The adapter converts DSH session events into a trace for 𝒥R and 𝒥L.
Figure 3: A.I.G runtime adapter for DSH. A.I.G prepares a tainted artifact and controlled fixture files. The real DSH TypeScript runtime handles the agent turn. The adapter converts DSH session events into a trace for 𝒥R and 𝒥L.
Table 2: Configuration of the assessed DSH deployment.
ItemConfiguration
RuntimeDeepSeek Harness TypeScript runtime
Base cases1,120 cases over 16 channels, 2 modes, and 35 objectives
Attack suite13 methods: naive plus 12 attack methods
Agent executions14,560 controlled runs
Carrier modes560 text-mode and 560 file-mode base cases
Objectives32 sink-required and 3 output-only tasks
Fixtures6 source tools and 8 tracked simulated sinks
Model backenddeepseek-v4-flash [5] through a local proxy
EvaluatorsOnline 𝒥R and offline 𝒥L
Figure 4: DSH source-to-sink path. A tool result can become model-visible context, after which the model may select another tool. DSH also exposes pre-execution and deny-only guard hooks that a deployment can use to block risky calls.
Figure 4: DSH source-to-sink path. A tool result can become model-visible context, after which the model may select another tool. DSH also exposes pre-execution and deny-only guard hooks that a deployment can use to block risky calls.
Table 3: Outcome distribution over 14,560 DSH executions. Full and partial are mutually exclusive.
Outcome𝒥R𝒥L
Full success819 (5.6%)772 (5.3%)
Partial compliance298 (2.0%)1,060 (7.3%)
Broad influence (full + partial)1,117 (7.6%)1,832 (12.6%)
Failure / not reached13,443 (92.4%)12,719 (87.4%)
Judge error09 (0.1%)
Figure 5: Dataset composition. The base dataset balances 16 channels, two carrier modes, and 35 payload objectives. The attack suite uses naive as an unmodified baseline and compares it with 12 attack methods.
Figure 5: Dataset composition. The base dataset balances 16 channels, two carrier modes, and 35 payload objectives. The attack suite uses naive as an unmodified baseline and compares it with 12 attack methods.
Table 4: Outcomes by carrier mode. Full and partial values are percentages; sink calls are counts and overlap with the outcome categories.
𝒥R𝒥L
ModeRunsFullPartialFullPartial
Text7,2805.1%2.1%5.2%7.5%
File7,2806.2%2.0%5.4%7.0%
Figure 6: Overall and selected attack-method results. The left panel shows full, partial, and non-success outcomes. The right panel compares selected attack methods. The vertical axis reports full-success rate (%).
Figure 6: Overall and selected attack-method results. The left panel shows full, partial, and non-success outcomes. The right panel compares selected attack methods. The vertical axis reports full-success rate (%).
Table 5: Attack success rates (%) by attack method and carrier mode. naive is the unmodified baseline.
MethodRule TextRule FileLLM TextLLM File
naive (baseline)5.5%5.9%5.7%5.5%
escape8.6%9.3%9.3%9.3%
context_ignoring0.2%1.8%0.4%1.6%
fake_completion16.6%16.1%17.0%15.5%
combined1.2%2.9%1.2%2.1%
payload_splitting2.1%4.6%3.4%4.5%
obfuscation13.6%13.6%9.1%8.8%
prefix_injection3.8%5.5%5.0%4.5%
format_confusion5.0%6.8%5.0%5.0%
context_flooding2.7%4.1%1.8%2.5%
cross_channel3.0%4.1%5.0%4.3%
important_instructions2.9%3.8%3.9%4.3%
stealth_instruction0.9%1.8%1.1%2.1%
Table 6: 𝒥R full-success rates (%) by channel. Each channel-mode cell contains 455 runs.
ChannelTextFileChannelTextFile
calendar_event7.7%4.2%chat_message9.5%1.3%
code_comment3.5%3.5%config_file4.6%2.4%
document5.5%7.3%email_headers3.3%3.3%
log_file2.4%2.4%markdown1.8%2.9%
pdf_metadata6.4%3.3%skills14.3%16.0%
spreadsheet3.5%5.3%structured_data4.0%4.2%
transaction_record2.0%4.0%translation7.9%8.8%
unicode_hidden0.0%25.5%webpage5.1%4.4%
Table 7: Key local artifacts used for the DSH case study.
ArtifactRole
A.I.G assessment componentsPayload construction, taint routing, and trace normalization
𝒥R / 𝒥LDeterministic and semantic outcome evaluation
full_channel_mode dataset1,120-case text/file channel matrix
experiment runnerMatrix orchestration and online 𝒥R evaluation
DSH runtime adapterPython-to-TypeScript harness bridge and trace mapping
DSH driverReal DSH runtime initialization and session-event export
controlled test pluginSource fixtures and tracked simulated sinks
final trace corpusOne normalized trace for each planned agent run
offline judge corpusTrace-level 𝒥L verdicts

为什么重要

这项研究对所有让AI智能体读取外部文档、邮件或网页内容的开发者都有直接警示意义,它用具体数字而非抽象风险说明了问题所在。它表明系统安全不仅取决于底层模型本身,还取决于整个智能体系统如何处理不可信内容以及如何授权后续操作。

本文术语

  • 间接提示注入(indirect prompt injection) · 把恶意指令藏在AI读取的内容中(而非用户直接输入)来操控其行为的攻击方式
  • 源到汇路径(source-to-sink) · 从攻击者控制的内容进入系统(源)到真正执行敏感操作(汇)之间的路径
  • 带毒内容(tainted content) · 被篡改后含有隐藏恶意指令的内容
  • J_R / J_L 判定器 · 判断攻击是否成功的两种方式,一种基于固定规则,一种用AI理解语义来判断
  • 技能(skills)渠道 · 供智能体复用的指令模块,本身也可能成为攻击入口

论文原文摘要（英文）

We assess indirect prompt injection in DeepSeek Harness (DSH), using AI-Infra-Guard (A.I.G) to construct tests, deliver controlled taint, execute DSH, collect traces, and judge outcomes. The study covers 14,560 controlled executions over 16 indirect-content channels, text and file carrier modes, 35 payload objectives, one unmodified baseline, and 12 attack methods. The experiment preserves DSH's agent loop, tool registry, model adapter, and session-event path; source tools and sensitive sinks are local fixtures, so attempted actions are recorded without external side effects. We evaluate each trace with a deterministic rule-based judge, (RuleJudge), and a semantic LLM-based judge, (LLMJudge). The strongest observed attack success rates are 17.0% under for fake-completion attack in text mode, 25.5% under for hidden Unicode in file mode, and 16.0% under for the skills channel in file mode. also assigns partial compliance more often than (7.3% versus 2.0%). We relate these results to DSH's treatment of tool results, additional contexts, and tool-call policy hooks, then identify controls that should sit between untrusted content and sensitive actions. Our code is available at https://github.com/Tencent/AI-Infra-Guard/tree/main/Research/deepseek-harness-security-assessment .

作者 · Zonghao Ying

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Zonghao Ying et al., arXiv:2608.16393, CC BY 4.0