UNMASK：发现并因果验证文本分类器中的虚假捷径

摘要

在大型众包语料库上训练的神经语言模型经常利用与目标标签相关的伪表面模式，这些模式不具备真正的语言学或因果相关性，从而在提升基准测试性能的同时，在面对对抗性或分布外输入时表现不佳。现有方法要么需要手动指定特征词汇表，要么仅部分自动化了发现过程，因而未能解决数据集层面相关性与模型层面利用之间的差距。我们提出了U N M ASK，一个全自动流水线，能够在无需额外人工标注的情况下，发现、因果验证并缓解文本分类器中的伪相关性。给定无标注训练样本，U N M ASK将候选表面模式生成为可执行的布尔表达式，通过具有独立重复的统计验证协议对它们进行筛选，并通过经核验的反事实干预确立因果模型依赖性。随后，经因果确认的特征可用作深度特征重新加权（Deep Feature Reweighting, DFR）的无标注组定义，从而免除了标准DFR所需的组标签。我们将该流水线应用于在MNLI上训练的BERT和RoBERTa，独立重新发现了已确立的词汇重叠和否定偏差，在BERT上验证了10个特征中的9个，在RoBERTa上验证了6个，并将HANS准确率提升了多达12.58个百分点。在CivilComments-WILDS上，程序化组在没有人口统计学标注的情况下，匹配了手工标注DFR（Kirichenko等人，2023年）70.1%的最差组准确率。我们进一步证明，发现和验证阶段可推广至奖励模型偏好数据，在RewardBench2中揭示了可解释的伪相关性。

English

Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs. Existing approaches either require manual specification of the feature vocabulary or automate discovery only partially, leaving the gap between dataset-level correlation and model-level exploitation unaddressed. We present U N M ASK, a fully automated pipeline that discovers, causally verifies, and mitigates spurious correlations in text classifiers without additional human annotation. Given unlabeled training examples, U N M ASK generates candidate surface patterns as executable boolean expressions, filters them through a statistical validation protocol with independent replication, and establishes causal model dependence via verified counterfactual interventions. Causally confirmed features then serve as annotation-free group definitions for Deep Feature Reweighting, eliminating the group labels that standard DFR requires. Applied to BERT and RoBERTa trained on MNLI, our pipeline independently rediscovers established lexical-overlap and negation biases, verifying 9 of 10 features on BERT and 6 on RoBERTa, and improving HANS accuracy by up to 12.58 pp. On CivilComments-WILDS, programmatic groups match the 70.1% worst- group accuracy of hand-labeled DFR (Kirichenko et al., 2023) without demographic annotation. We further demonstrate that the discovery and validation stages generalize to reward model preference data, surfacing interpretable spurious correlations in RewardBench2.