评分标准作为攻击面：LLM评审中隐蔽的偏好漂移

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

February 14, 2026

作者: Ruomeng Ding, Yifei Pang, He Sun, Yizhong Wang, Zhiwei Steven Wu, Zhun Deng

cs.AI

摘要

针对大语言模型的评估与对齐流程日益依赖基于LLM的评判器，其行为由自然语言量规引导并通过基准测试进行验证。我们发现该工作流中存在一个先前未被充分认识的脆弱性，称之为"量规诱导偏好漂移"。即使量规修改通过了基准验证，仍可能导致评判器在目标领域产生系统性、方向性的偏好偏移。由于量规作为高层决策接口，此类漂移可能源于看似自然且保持评判标准的修改，并通过聚合基准指标或有限抽样检查难以察觉。我们进一步证明该脆弱性可通过基于量规的偏好攻击被利用——符合基准测试的量规修改会使目标领域的判断偏离固定的人类或可信参照标准，系统性地诱发RIPD现象，导致目标领域准确率最高下降9.5%（实用性）和27.9%（无害性）。当这些判断结果被用于生成下游训练所需的偏好标签时，诱导的偏差会通过对齐流程传播并内化至训练后的策略中，最终导致模型行为出现持续系统性的偏移。总体而言，我们的研究揭示了评估量规作为敏感且可操纵的控制接口，展现了一种超越评估器可靠性的系统级对齐风险。代码已开源：https://github.com/ZDCSlab/Rubrics-as-an-Attack-Surface。警告：部分内容可能包含不适宜所有读者的潜在有害信息。

English

Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-language rubrics and validated on benchmarks. We identify a previously under-recognized vulnerability in this workflow, which we term Rubric-Induced Preference Drift (RIPD). Even when rubric edits pass benchmark validation, they can still produce systematic and directional shifts in a judge's preferences on target domains. Because rubrics serve as a high-level decision interface, such drift can emerge from seemingly natural, criterion-preserving edits and remain difficult to detect through aggregate benchmark metrics or limited spot-checking. We further show this vulnerability can be exploited through rubric-based preference attacks, in which benchmark-compliant rubric edits steer judgments away from a fixed human or trusted reference on target domains, systematically inducing RIPD and reducing target-domain accuracy up to 9.5% (helpfulness) and 27.9% (harmlessness). When these judgments are used to generate preference labels for downstream post-training, the induced bias propagates through alignment pipelines and becomes internalized in trained policies. This leads to persistent and systematic drift in model behavior. Overall, our findings highlight evaluation rubrics as a sensitive and manipulable control interface, revealing a system-level alignment risk that extends beyond evaluator reliability alone. The code is available at: https://github.com/ZDCSlab/Rubrics-as-an-Attack-Surface. Warning: Certain sections may contain potentially harmful content that may not be appropriate for all readers.