LLM作為驗證器：通用驗證框架

摘要

扩展预训练、后训练和测试时计算已成为提升大语言模型性能的核心范式。在本研究中，我们将验证——即判断解决方案正确性的能力——确定为新的扩展维度。为突破这一限制并展示其有效性，我们提出了“LLM-as-a-Verifier（作为验证器的LLM）”框架，这是一个通用的验证框架，能够在不需额外训练的情况下为智能体任务提供细粒度反馈。与标准的大语言模型评判方式（即通过提示LLM为候选方案生成离散分数）不同，LLM-as-a-Verifier通过计算评分词元逻辑值分布上的期望来生成连续分数。这种概率化表述使得验证能够在多个维度上扩展：（1）评分粒度，（2）重复评估，以及（3）标准分解。特别是，我们展示了扩展评分粒度能够更好地区分正面与负面解决方案，从而产生更经校准的比较结果。此外，通过降低方差与复杂度，扩展重复评估与标准分解能持续提升验证精度。我们进一步提出了一种成本高效的排序算法，利用验证器的连续分数从候选方案中选出最佳方案。LLM-as-a-Verifier在Terminal-Bench V2（86.5%）、SWE-Bench Verified（78.2%）、RoboRewardBench（87.4%）和MedAgentBench（73.3%）上取得了最先进的性能。除验证之外，LLM-as-a-Verifier生成的细粒度信号还可作为估计任务进展的代理。我们为Claude Code构建了一个扩展，使开发者能够监控并改进自己的智能体系统。最后，我们展示了LLM-as-a-Verifier能够为强化学习提供密集反馈，从而提高SAC与GRPO在机器人与数学推理基准上的样本效率。

English

Scaling pre-training, post-training, and test-time compute have become the central paradigms for improving the capabilities of LLMs. In this work, we identify verification, the ability to determine the correctness of a solution, as a new scaling axis. To unlock this and demonstrate its effectiveness, we introduce LLM-as-a-Verifier, a general-purpose verification framework that provides fine-grained feedback for agentic tasks without requiring additional training. Unlike standard LM judges that prompt LLMs to produce discrete scores for candidate solutions, LLM-as-a-Verifier computes the expectation over the distribution of scoring token logits to generate continuous scores. This probabilistic formulation enables verification to scale along multiple dimensions: (1) score granularity, (2) repeated evaluation, and (3) criteria decomposition. In particular, we show that scaling the scoring granularity leads to better separation between positive and negative solutions, resulting in more calibrated comparisons. Moreover, scaling repeated evaluation and criteria decomposition consistently lead to additional gains in verification accuracy through variance and complexity reduction. We further introduce a cost-efficient ranking algorithm for selecting the best solution among candidates using the verifier's continuous scores. LLM-as-a-Verifier achieves state-of-the-art performance on Terminal-Bench V2 (86.5%), SWE-Bench Verified (78.2%), RoboRewardBench (87.4%), and MedAgentBench (73.3%). Beyond verification, the fine-grained signals from LLM-as-a-Verifier can also serve as a proxy for estimating task progress. We build an extension for Claude Code, enabling developers to monitor and improve their own agentic systems. Finally, we show that LLM-as-a-Verifier can provide dense feedback for RL, improving the sample efficiency of SAC and GRPO on robotics and mathematical reasoning benchmarks.