VeriEvol：透過可驗證演化指令擴展多模態數學推理

摘要

扩展视觉数学推理的强化学习不仅需要生成更难的题目：随着数据量增长，奖励标签本身必须保持可靠。然而，现有的数据流水线在扩展监督能力时依赖标注者的可信度，而策略层面的方法则假设基础答案已经正确。我们转而将扩展视为一个可验证的数据构建问题，并在任何策略更新之前，将两个维度解耦：提示难度（通过路径特定的演化算子扩展）和答案可靠性（通过离线假设检验的证伪机制强制保证）。我们将其实现为VeriEvol框架，这是一个迭代框架，包含两个可扩展组件：一个类型感知的演化模块，将低难度的图文种子重写为更难的、基于图像的提示；以及HTV-Agent验证器，该验证器仅在多方反证均未能反驳答案时，才接受该答案。由此生成的经过验证的数据，其在规模上可扩展，可通过增加演化路径或验证通道进行扩展，并可直接接入现有的GRPO样式的强化学习方案中。在一个包含五个基准的视觉数学测试集上，将已演化的SFT数据从10K样本扩展到250K样本，可使平均准确率从35.42提升至54.73；随后，在保持主干网络、SFT初始化以及GRPO方案不变的情况下，VeriEvol在未演化的强化学习基线之上累计提升了+3.88个百分点，其中+1.82来自演化后的提示，+2.06来自HTV-Agent验证器。我们开源了提示、数据、模型、代码以及每个样本的完整验证轨迹，以便后续研究能够扩展和审计整个流水线，而不仅仅是检查其输出。

English

Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions: as data volume grows, the reward labels themselves must remain reliable. Yet existing data pipelines scale supervision while trusting the labeller, and policy-side methods assume the underlying answers are already correct. We instead treat scaling as a verifiable data-construction problem and decouple two axes before any policy update: prompt difficulty, expanded by route-specific evolution operators, and answer reliability, enforced by offline hypothesis-test falsification. We instantiate this as VeriEvol, an iterative framework with two extensible components: a type-aware evolution module that rewrites low-difficulty image-question seeds into harder, image-grounded prompts; and HTV-Agent, a verifier that accepts an answer only after multi-source counter-evidence has failed to refute it. The resulting verified data scales in volume, extends by adding evolution routes or verifier channels, and plugs directly into existing GRPO-style RL recipes. On a five-benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73; then, with backbone, SFT initialization, and GRPO recipe held fixed, VeriEvol adds a cumulative +3.88 over an un-evolved RL baseline, of which +1.82 comes from evolved prompts and +2.06 from the HTV-Agent verifier. We release the prompts, data, models, code, and the full verifier trace of every sample, so that downstream work can scale and audit the pipeline rather than only inspect its outputs.