UNMASK：在文本分類器中發現並因果驗證虛假捷徑

摘要

在大型群眾外包語料庫上訓練的神經語言模型，經常利用與目標標籤相連的虛假表面模式，而這些模式並無真正的語言學或因果關聯；這使得基準評測表現提升，但在對抗性或分布外輸入上卻會失敗。現有方法或需手動指定特徵詞彙表，或僅部分自動化發現流程，因而未能解決資料集層級相關性與模型層級利用之間的鴻溝。我們提出 UNMASK，一個全自動化流程，可在無需額外人工標註的情況下，發現、因果驗證並緩解文本分類器中的虛假相關。給定未標註的訓練樣本，UNMASK 生成以可執行布林表達式呈現的候選表面模式，透過具獨立重複的統計驗證協議進行過濾，並藉由經驗證的反事實干預建立因果模型依賴性。經因果確認的特徵隨後作為無需標註的群組定義，用於深度特徵重新加權（Deep Feature Reweighting, DFR），從而消除標準 DFR 所需的群組標籤。應用於在 MNLI 上訓練的 BERT 與 RoBERTa 時，我們的流程獨立重新發現了已確立的詞彙重疊與否定偏誤，在 BERT 上驗證了 10 個特徵中的 9 個，在 RoBERTa 上驗證了 6 個，並將 HANS 準確率提升了最多 12.58 個百分點。在 CivilComments-WILDS 上，程式化群組達到了與人工標註 DFR（Kirichenko et al., 2023）相同的 70.1% 最差群組準確率，且無需人口統計標註。我們進一步證明，發現與驗證階段可推廣至獎勵模型的偏好資料，在 RewardBench2 中揭示出可解釋的虛假相關。

English

Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs. Existing approaches either require manual specification of the feature vocabulary or automate discovery only partially, leaving the gap between dataset-level correlation and model-level exploitation unaddressed. We present U N M ASK, a fully automated pipeline that discovers, causally verifies, and mitigates spurious correlations in text classifiers without additional human annotation. Given unlabeled training examples, U N M ASK generates candidate surface patterns as executable boolean expressions, filters them through a statistical validation protocol with independent replication, and establishes causal model dependence via verified counterfactual interventions. Causally confirmed features then serve as annotation-free group definitions for Deep Feature Reweighting, eliminating the group labels that standard DFR requires. Applied to BERT and RoBERTa trained on MNLI, our pipeline independently rediscovers established lexical-overlap and negation biases, verifying 9 of 10 features on BERT and 6 on RoBERTa, and improving HANS accuracy by up to 12.58 pp. On CivilComments-WILDS, programmatic groups match the 70.1% worst- group accuracy of hand-labeled DFR (Kirichenko et al., 2023) without demographic annotation. We further demonstrate that the discovery and validation stages generalize to reward model preference data, surfacing interpretable spurious correlations in RewardBench2.