Are LLMs Safe Beyond Text: Do Emojis Expose Gaps in Safety Evaluation

arXiv:2608.181642026-08-20

用表情符号提问,可能绕过AI的安全防线

这项研究用夹杂表情符号的提示词而非纯文本,测试了四个开源大语言模型(Mistral 7B、Qwen 2 7B、Gemma 2 9B、Llama 3 8B)的安全机制。将同样的50条表情符号提示词分别输入各模型后,发现不同模型被攻破的程度差异很大。这说明只靠纯文本做安全评估,可能会漏掉这类输入形式带来的风险。

论文原文摘要（英文）

Safety evaluations of large language models (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arising from alternative input representations. This work examines emoji-augmented prompts as a test case for this gap, evaluating 50 prompts across four open-source LLMs (Mistral 7B, Qwen 2 7B, Gemma 2 9B, Llama 3 8B). Results show substantial variation in robustness: Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%), Llama 3 8B 6%, while Qwen 2 7B shows complete resistance (0% success rate). A chi-square test ($\chi^2 = 32.94, p < 0.001$) confirms significant differences in outcome distributions. These findings indicate that robustness is sensitive to input representation, and that evaluations restricted to standard text prompts may underrepresent model vulnerabilities.

作者 · M P V S Gopinadh