Abliteration Mitigation via Refusal Aliases

arXiv:2608.180932026-08-20

新方法AMRA把AI模型里的拒绝方向藏起来，防止被abliteration攻击一键抹除

语言模型拒绝有害请求的能力其实集中在内部激活空间的一个方向上，攻击者只需少量对比提示就能找到并抹除这个方向，这种手法叫abliteration。AMRA通过修改少量权重矩阵，把触发拒绝的激活值替换成随机噪声，让这个方向难以被提取，从而抵御攻击。在Llama-3-8B上几乎不损失模型性能就实现了防御，在Gemma-2-9B上防御效果更强，但代价是性能有所下降。

METAL MEDIA 解读图

新方法AMRA把AI模型里的拒绝方向藏起来，防止被abliteration攻击一键抹除

01以往的防御方法大多没有关注拒绝方向为何如此容易被提取，这项工作直接针对提取过程本身下手
02AMRA对向残差流写入信息的少数权重矩阵做rank-k（低秩）更新，把触发拒绝的激活替换为随机低方差的替代值，并修正下游读取权重使模型其余行为保持不变
03在Llama-3-8B上，相比未加防护的基线模型，AMRA让abliteration攻击后的拒绝得分提高了2.16分，MMLU性能下降不到0.5个百分点
04在Gemma-2-9B上，AMRA让攻击后的拒绝得分比基线提高14.70分，有害输出比例与基线相近，但GSM8K和MMLU等实用性指标下降更明显
05与Surgical、CAST、Circuit Breakers、AlphaSteer等现有防御方法相比，AMRA是唯一能同时提升攻击后拒绝得分、保持较低的HarmBench攻击成功率和LlamaGuard不安全率，并维持较好实用性的方法

这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

以往的防御方法大多没有关注拒绝方向为何如此容易被提取，这项工作直接针对提取过程本身下手
AMRA对向残差流写入信息的少数权重矩阵做rank-k（低秩）更新，把触发拒绝的激活替换为随机低方差的替代值，并修正下游读取权重使模型其余行为保持不变
在Llama-3-8B上，相比未加防护的基线模型，AMRA让abliteration攻击后的拒绝得分提高了2.16分，MMLU性能下降不到0.5个百分点
在Gemma-2-9B上，AMRA让攻击后的拒绝得分比基线提高14.70分，有害输出比例与基线相近，但GSM8K和MMLU等实用性指标下降更明显
与Surgical、CAST、Circuit Breakers、AlphaSteer等现有防御方法相比，AMRA是唯一能同时提升攻击后拒绝得分、保持较低的HarmBench攻击成功率和LlamaGuard不安全率，并维持较好实用性的方法

Figure 1: Obfuscation diagram. We show a high-level depiction of the patches made to obfuscate the refusal signal. Although our weight matrix updates are rank-k, we show a rank-one variant for brevity. The bottom arrow represents the residual stream through layer l. The Q, K, and V, up projections into the attention mechanism, downward projection from the feed-forward, and LayerNorm have been omitted to highlight sublayer interactions and general stylistic simplicity.

Table 1: Safety and abliteration results. Refusal scores are higher when the model retains more refusal behavior before and after Arditi-style abliteration. Additionally, the Arditi abliteration on defenses other than our baseline. (None) implies the difference-in-means refusal vector extraction was run again after a defense was applied. HarmBench ASR and LlamaGuard unsafe rate are lower when the model is safer.
Model	Defense	Clean Refusal ↑	Arditi ↑	HarmBench ASR ↓	LlamaGuard ↓
Llama-3-8B	None	10.0318	5.6884	0.0200	0.0100
	AMRA	10.2350	7.8497	0.0100	0.0100
	Surgical	1.3365	2.0229	0.4200	0.2800
	CAST	-0.2591	-0.5980	0.0000	0.8300
	CB	9.9183	5.5355	0.0200	0.0400
	AlphaSteer	10.0184	5.6796	0.0200	0.0200
Gemma-2-9B	None	7.1172	-16.0093	0.0200	0.0000
	AMRA	7.2000	-1.3105	0.0100	0.0000
	Surgical	-16.9482	-18.0443	0.4400	0.7300
	CAST	7.3802	-12.3973	0.0000	0.0000
	CB	7.1436	-16.0190	0.0200	0.0000
	AlphaSteer	5.6320	0.8756	0.0900	0.1100

Figure 2: Residual stream shifts. Here, we show the effect of a rank-one (kw=1) update of a writer matrix Woutl on subsequent residual stream representations. Specifically, we compare the mean residual stream values prior to and after the weight matrix update over 16 prompts. (a) exhibits the L2 difference, (b) shows the shift in residual stream variance, and (c) shows the cosine similarity of the residual stream before versus after the edit. Upper: sweeps over ε for layer 20. Lower: sweeps over layers along with an additional random selection of 6.

Table 2: Utility results across base models and defenses. Lower BPB is better; higher GSM8K and MMLU are better.
Model	Defense	Pile BPB ↓	Alpaca BPB ↓	GSM8K ↑	MMLU ↑
Llama-3-8B	None	0.7665	0.5555	0.7020	0.6911
	AMRA	0.7801	0.5651	0.6880	0.6876
	Surgical	0.7434	0.5141	0.7420	0.6782
	CAST	1.2924	0.8295	0.0320	0.3334
	CB	0.7674	0.5576	0.7080	0.6825
	AlphaSteer	0.7667	0.5555	0.6980	0.6912
Gemma-2-9B	None	0.8124	0.6561	0.5620	0.7411
	AMRA	0.9597	0.6679	0.3360	0.6926
	Surgical	0.8577	0.6602	0.6760	0.6971
	CAST	0.8117	0.6526	0.5080	0.7393
	CB	0.8124	0.6563	0.5620	0.7412
	AlphaSteer	1.5838	1.0017	0.1680	0.6611

为什么重要

目前任何拿到开源模型权重的人都能用极少量提示词抹掉模型的安全拒绝能力，这项研究为模型开发者在发布权重前加固模型提供了一种实用手段。但作者也坦言，如果攻击者能拿到未加保护的原始权重，这种防御就会失效，这也划定了它的适用边界。

本文术语

abliteration · 将模型权重矩阵投影到与拒绝方向正交，从而清除模型拒绝能力的技术
残差流（residual stream） · transformer内部各层读取和写入信息所共享的向量通路
rank-k更新 · 只在少数几个方向（k个）上修改权重矩阵的低维改动方式
difference-in-means · 通过计算有害提示与无害提示激活值的平均差异来提取方向向量的方法
HarmBench ASR / LlamaGuard unsafe rate · 分别衡量有害提示攻击成功率和输出被判定为不安全比例的安全性指标

论文原文摘要（英文）

Abliteration, the removal of refusal capabilities from large language models by projecting weight matrices orthogonal to an extracted refusal direction, has emerged as a prominent safety concern through its ability to bypass post-training alignment using only a small set of contrastive prompts. We find that existing defenses commonly overlook the cause of abliteration; that is, how easily the refusal direction can be extracted. To hinder this process, we introduce a weight-editing method that obscures the refusal signal by applying rank-$k$ updates to residual stream writer matrices while replacing refusal-inducing activations with random aliases and correcting downstream reader matrices to preserve the model's original behavior. On Llama-3-8B, AMRA improves post-abliteration refusal scores by $2.16$ points over the undefended baseline with less than $0.5$ percentage points of MMLU degradation. On Gemma-2-9B, it improves the post-abliteration refusal by $14.70$ points over the baseline while keeping harmful output rates similar to the baseline, albeit at a greater utility cost.

作者 · Nathan Truong

在 arXiv 阅读

METAL MEDIA 最新报道

图片来源： Nathan Truong et al., arXiv:2608.18093, CC BY 4.0

Abliteration Mitigation via Refusal Aliases

新方法AMRA把AI模型里的拒绝方向藏起来，防止被abliteration攻击一键抹除

他们做了什么

为什么重要

本文术语

最新论文

METAL MEDIA 最新报道