当多个答案均有效时，投票失效：大语言模型中Best-of-K因果推理的符号验证

摘要

自一致性假设采样推理轨迹中最频繁的答案最为可靠，但这一假设在因果推理中可能失效：样本往往重复相同的混杂错误，且投票分散在多个有效答案之间，导致无效答案胜出，而有效的少数轨迹被忽略。我们提出了CALVER（Causal Axiom-Level VERification，因果公理级验证），一种无需训练的符号验证器，它依据Pearl因果准则（包括d分离、后门调整和干预）对结构化轨迹进行评分，并在不参考参考答案的情况下选择得分最高的候选。在CLEAR数据集上允许多个图有效答案的“查找单一有效答案”查询中，CALVER达到42.1%的准确率，而在相同的冻结候选池上，多数投票、奖励模型、LLM评判器和模型置信度均维持在30%左右。将评判器扩展到720亿参数也未能缩小这一差距。在经审计的干净核心子集中，21个图有效的CALVER选择中有11个不同于基准测试列出的答案，但仍满足所要求的谓词。该优势随采样预算增加而扩大，并在十个已发表的贝叶斯网络、第二个模型系列以及模型必须从文本构建图的设置中得以复现。CALVER还改善了基于精确真值的阈值化平均处理效应决策，在真值表检查器下可推广到逻辑推理，并在CPU上以毫秒级速度为每个候选评分。CALVER仅需因果结构——可直接提供或从文本中构建；只要满足这一条件，选择即可通过因果有效性进行聚合。

English

Self-consistency assumes the most frequent answer among sampled reasoning traces is the most reliable, but this can fail in causal reasoning: samples often repeat the same confounding error, and votes fragment across multiple valid answers, letting an invalid answer win despite a valid minority trace. We introduce CALVER (Causal Axiom-Level VERification), a training-free symbolic verifier that scores structured traces against Pearl's causal criteria, including -separation, backdoor adjustment, and intervention, and selects the highest-scoring candidate without consulting a reference answer. On CLEAR find-one-valid queries that admit multiple graph-valid answers, CALVER reaches 42.1% where plurality, a reward model, an LLM judge, and model confidence remain near 30% on identical frozen pools. Scaling the judge to 72B does not close the gap. In an audited clean-core subset, 11 of 21 graph-valid CALVER selections differ from the benchmark's listed answer while still satisfying the requested predicate. The advantage widens with the sampling budget and reproduces across ten published Bayesian networks, a second model family, and settings where the model must build the graph from text. CALVER also improves thresholded average-treatment-effect decisions against exact ground truth, generalizes to logic under a truth-table checker, and scores each candidate in milliseconds on CPU. CALVER needs only a causal structure, supplied outright or built from the text; wherever that holds, selection can aggregate via causal validity.