感知准则：校准多模态评估以契合人类感知

摘要

我们提出 PerceptionRubrics，一种基于评定量表的评估框架，旨在弥合基准测试分数饱和与现实世界脆弱性之间的鸿沟。该框架将评估从整体语义匹配转向严格的原子化审核，为 1038 张信息密集的图像配对了超过 12000 条实例特定的评定量表。这些标准源自通过新颖的循环同行评审共识流程构建的金标准描述，并进一步提炼为“必须正确”（基本事实）与“容易错误”（细粒度细节）的双流评定量表系统。关键在于，PerceptionRubrics 实现了门控评分机制：与线性平均值不同，一旦对强制性视觉事实的检验失败，便会触发尖锐的二元惩罚。大量评估揭示了关键洞见：（1）可靠性差距：模型通常能正确验证碎片化元素，但在严格的合取约束下却失败，暴露了密集领域中的脆弱性；（2）开放-封闭分层：与推理趋势相反，我们揭示了开源前沿与专有前沿之间持续存在的 8% 感知缺陷；（3）人类对齐的严格性：我们的门控指标显著优于传统基准的对齐程度，验证了严格的感知保真度是可靠生成的前提条件。

English

We introduce PerceptionRubrics, a rubric-based evaluation framework that addresses the gap between saturated benchmark scores and real-world brittleness. Shifting evaluation from holistic semantic matching to rigorous atomic auditing, PerceptionRubrics pairs 1,038 information-dense images with over 12,000 instance-specific rubrics. These criteria are derived from golden captions constructed via a novel Circular Peer-Review consensus pipeline and then distilled into a dual-stream system of Must-Right (essential facts) and Easy-Wrong (fine-grained details) rubrics. Crucially, PerceptionRubrics implements a Gated Scoring mechanism: unlike linear averages, failure on mandatory visual facts triggers sharp binary penalties. Extensive evaluation yields critical insights: (1) The Reliability Gap: models often verify fragmented elements correctly yet fail strict conjunctive constraints, exposing brittleness in dense domains; (2) Open-Closed Stratification: contrary to reasoning trends, we reveal a persistent 8% perception deficit between open-source and proprietary frontiers; and (3) Human-Aligned Rigor: our gated metrics substantially out-align conventional benchmarks, validating that strict perceptual fidelity is the prerequisite for reliable generation.