当上下文反噬：通过文档级注意力崩溃检测RAG投毒

摘要

检索增强生成（RAG）对于增强大语言模型不可或缺。然而，RAG系统日益容易受到投毒攻击，即注入对抗性文档以操纵生成器的输出。以往的方法依赖于输出端信号（如困惑度和一致性检查）来检测此类攻击。然而，我们的分析表明，蓄意攻击往往引发虚假自信：被投毒输出的困惑度甚至低于良性输出，这使得基于不确定性的检测失效。为应对这一挑战，我们探索了生成器的内部动态，并识别出一种独特特征，即“注意力坍缩”（Attention Collapse）。与良性生成中注意力分散的情况不同，受攻击生成的注意力集中于被投毒文档，导致熵降低。基于这些发现，我们提出了D-SCAN（文档级信号坍缩分析），一种通过监测注意力动态来识别受攻击生成结果的轻量级检测框架。在多个攻击基准上的大量实验验证了我们方法的有效性。此外，即使攻击未能改变最终答案，D-SCAN也能检测到攻击。代码可在 https://github.com/yingtaoren/D-Scan.git 获取。

English

Retrieval-augmented generation (RAG) is indispensable for enhancing large language models. However, RAGs are increasingly susceptible to poisoning attacks, in which adversarial documents are injected to manipulate generator outputs. Previous methods rely on output-side signals such as perplexity and consistency checks to detect such attacks. Nevertheless, our analysis reveals that deliberate attacks often induce false confidence, where poisoned outputs exhibit even lower perplexity than benign ones, rendering uncertainty-based detection ineffective. To address this challenge, we explore the internal dynamics of the generator and identify a distinctive signature termed Attention Collapse. Unlike the dispersed attention in benign generations, attacked generations exhibit a decrease in entropy as attention concentrates on poisoned documents. Building on these findings, we propose D-SCAN (Document-level Signal Collapse Analysis), a lightweight detection framework that monitors attention dynamics to identify attacked generations. Extensive experiments on multiple attack benchmarks demonstrate the effectiveness of our method. Moreover, D-SCAN can detect attacks even when they fail to alter the final answer. Code is available at https://github.com/yingtaoren/D-Scan.git.