V-Rubrics：基于评分准则强化学习的视觉忠实度

摘要

视觉语言模型可能生成流畅的答案，但这些答案在视觉证据上的依据往往不足：一个不受支持的物体、图表数值或中间推理步骤，就可能使原本合理的回答大打折扣。我们认为这属于多模态后训练中的信用分配失败。标量结果奖励只能指示答案是否可接受，却无法识别哪些视觉事实有依据、哪些推理步骤有效，以及哪些指令约束被遗漏。我们提出了基于视觉评分标准的强化学习（Visual Rubrics-Based Reinforcement Learning），该方法将参考回答分解为原子命题，并沿视觉忠实度（VF）、推理一致性（RC）和指令遵循（IF）三个维度对生成答案进行评分。由此产生的评分标准条目提供结构化的部分信用，并在有支持性证据片段可用时实现评分标准信用的定位。我们首先在公开的 OpenMMReasoner-SFT-874K 语料库上微调 Qwen3-VL-8B-Instruct，并沿用 OpenMMReasoner 的冷启动数据方案，从而获得一个 SFT 检查点。我们构建了 V-Rubrics 50K——一个包含 50,248 个样本、来自 17 个具视觉依据来源的训练集：先应用基于规则的过滤器，再根据拒绝采样分数推导样本难度，最后使用 Gemini-3-Pro 在相同的结构化提示和协议下对每个样本进行标注。我们基于相同的 SFT 检查点，使用逐组件、前缀定位的评分标准信用来训练模型。实验表明，我们基于评分标准的 GRPO 在共享的 SFT 基线和仅使用答案的 GRPO 之上均有提升，在知识导向和具视觉依据的推理基准上收益最大。结果表明，评分标准是一种对视觉后训练有用的奖励抽象。

English

Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.