验证视野：编码智能体奖励并无银弹

摘要

經典的直覺認為，驗證解決方案比提出方案更容易。但對當今的程式設計代理而言，這種直覺正在被顛覆：隨著基礎模型推理能力持續增強、工程架構日益複雜，生成複雜的候選解決方案已不再困難——真正棘手的反而是可靠地驗證它們。我們所能建構的一切驗證器，都只是人類意圖的代理，而非意圖本身。這使得驗證面臨雙重困境：首先，意圖本質上欠指定，難以忠實地檢驗其是否被滿足；其次，在模型訓練過程中，最佳化會擴大代理與意圖之間的差距——表現為獎勵作弊或信號飽和。為解決此問題，我們沿三個維度——可擴展性、忠實性與穩健性——來刻畫驗證信號的品質，並主張同時達成三者是核心挑戰。我們進一步研究了四種獎勵建構方式：用於通用程式設計任務的測試驗證器、用於前端任務的評分量規驗證器、用於真實世界代理任務的使用者即驗證器，以及用於長時域任務的自動化代理驗證器。針對不同任務類型與策略能力層級，我們深入分析並實驗了獎勵設計的核心難題，以及如何更有效地利用獎勵信號。實驗表明，針對性的驗證設計能有效抑制獎勵作弊、提升任務完成品質，並在多個內部與公開基準測試中取得顯著增益。這些經驗共同指向一個核心觀察：沒有任何固定的獎勵函數能在策略能力持續成長時保持有效；驗證必須與生成器共同演化。

English

A classical intuition holds that verifying a solution is easier than producing one. For today's coding agents, this intuition is being inverted: as foundation models develop stronger reasoning capabilities and engineering harnesses grow more sophisticated, generating complex candidate solutions is no longer difficult -- reliably verifying them has become the harder problem. Every verifier we can build is only a proxy for human intent, never the intent itself. This makes verification subject to a twofold difficulty: first, intent is underspecified by nature, making it inherently hard to faithfully check whether it has been fulfilled; second, during model training, optimization widens the gap between proxy and intent -- manifesting as reward hacking or signal saturation. To address this, we characterize the quality of verification signals along three dimensions -- scalability, faithfulness, and robustness -- and argue that achieving all three simultaneously is the central challenge. We further study four reward constructions: a test verifier for general coding tasks, a rubric verifier for frontend tasks, the user as verifier for real-world agent tasks, and an automated agent verifier for long-horizon tasks. Across different task types and policy capability levels, we conduct in-depth analysis and experiments on the core challenges of reward design and how to more effectively leverage reward signals. Experiments show that targeted verification design can effectively suppress reward hacking, improve task completion quality, and achieve significant gains across multiple internal and public benchmarks. These experiences collectively point to a core observation: no fixed reward function can remain effective as policy capability continues to grow; and verification must co-evolve with the generator.