Token級別幻覺檢測之時序多信號融合

摘要

詞元層級的幻覺偵測器根據單一訊號獨立地對每個詞元進行評分，而當生成模型自信地出錯時，它們便會失效。本研究則將幻覺視為一段在時間上延伸的區間，並透過序列標註加以偵測：每個詞元由一個 33 維的特徵流進行評分，該特徵流融合了文本統計、自然語言推論（NLI）蘊涵關係以及語言模型的驚奇度，且無需存取模型內部。在這些特徵上使用雙向門控循環單元（BiGRU），於 RAGTruth（10 種隨機種子）上達到 0.840 的 AUC，比獨立邏輯迴歸基線高出 11 個百分點（p = 0.002，威爾科克森符號秩檢驗）。一項受控分解分析將大部分增益歸因於時間順序而非模型容量：在區間內，證據會從高信心位置傳播至鄰近的模糊位置。相同的 0.845 上限反覆出現在循環、狀態空間（Mamba）與注意力架構中，顯示瓶頸在於特徵集而非模型本身。由於偵測器僅讀取生成的文本與外部訊號，它適用於閉源模型，且對於訓練期間從未見過的語言模型所生成的文本仍能保持有效，AUC 損失低於 4%。

English

Token-level hallucination detectors score each token independently from a single signal, and fail exactly when the generating model is confidently wrong. This paper instead treats hallucination as a temporally extended span and detects it by sequence labeling: each token is scored from a 33-dimensional feature stream that fuses text statistics, Natural Language Inference (NLI) entailment, and language model surprisal, with no access to model internals. A Bidirectional Gated Recurrent Unit (BiGRU) over these features reaches an AUC of 0.840 on RAGTruth (10 seeds), an 11-point gain over an independent logistic-regression baseline (p = 0.002, Wilcoxon signed-rank). A controlled decomposition attributes most of the gain to temporal order rather than model capacity: evidence propagates from confident positions to ambiguous neighbors within a span. The same 0.845 ceiling recurs across recurrent, state-space (Mamba), and attention architectures, locating the bottleneck in the feature set rather than the model. Because it reads only the generated text and external signals, the detector works on closed-source models, and it keeps working on text produced by language models it never saw during training, losing under 4% AUC.