解碼層級禁忌：大型語言模型穩健性的診斷性壓力測試

摘要

大型語言模型的評測通常聚焦於名義條件下的表現，製造出一種能力幻覺，使模型得以舒適地行走在一條狹窄且高度最佳化的生成走廊上。然而，在實際部署中，複雜的系統提示、安全護欄與結構性約束會持續迫使模型偏離這一名義路徑，導致基準分數與部署效能之間產生落差。為解決此問題，我們提出「解碼層級禁忌」（Decoding-Level Taboo），這是一種零提示診斷壓力測試，在執行時期直接介入 logit 空間，強迫模型脫離其名義路徑。透過在詞邊界動態遮罩主要候選詞元，Taboo 強迫機器進行迂迴表述。在多個開放權重模型家族上評估 Taboo 的結果顯示，路徑外穩健性深受參數規模與後訓練指令對齊的影響，且穩健性通常隨模型規模與對齊程度提升而增加。除了本文呈現的結果外，Taboo 亦提供了一種新穎的基礎工具，可用於生成多樣化的合成資料集、對執行時期的安全護欄進行壓力測試，以及在實際部署前審查模型可靠性。

English

Large language model evaluations typically focus on performance under nominal conditions, creating an illusion of capability where models comfortably walk a narrow, highly optimized generation corridor. In real-world deployments, however, complex system prompts, safety guardrails, and structural constraints continuously force models off this nominal path, driving a divergence between benchmark scores and deployment performance. To address this issue, we introduce Decoding-Level Taboo, a zero-prompt diagnostic stress test that intervenes directly in logit space at runtime, forcing models out of their nominal paths. By dynamically masking primary candidate tokens at word boundaries, Taboo forces machine circumlocution. Evaluating Taboo across several open-weight model families reveals that off-path robustness is heavily influenced by both parameter scale and post-training instruction alignment, with robustness generally improving with model size and alignment. Beyond the results presented in this paper, Taboo provides a novel primitive for generating diverse synthetic datasets, stress-testing runtime safety guardrails, and auditing model reliability prior to real-world deployment.