PerceptionRubrics：人間の知覚に合わせたマルチモーダル評価の較正

要旨

PerceptionRubrics（感知评估准则）是一种基于评分标准的评估框架，旨在弥合饱和的基准分数与现实世界脆弱性之间的差距。该框架将评估从整体语义匹配转向严格的原子审计，将1,038张信息密集的图像与超过12,000个实例特定的评分标准配对。这些标准源自通过新颖的循环同行评审共识流程构建的黄金描述，随后被提炼为双流系统：必须正确（基本事实）和容易错误（细粒度细节）的评分标准。关键在于，PerceptionRubrics实现了门控评分机制：与线性平均值不同，在强制性视觉事实上的失败会触发严格的二元惩罚。广泛评估得出了重要见解：(1) 可靠性差距：模型通常能正确验证碎片化元素，但在严格的合取约束下失败，暴露了密集领域的脆弱性；(2) 开源与闭源分层：与推理趋势相反，我们揭示了开源与专有前沿之间持续存在的8%感知差距；(3) 人类对齐的严格性：我们的门控指标显著优于传统基准，验证了严格的感知保真度是可靠生成的前提条件。

English

We introduce PerceptionRubrics, a rubric-based evaluation framework that addresses the gap between saturated benchmark scores and real-world brittleness. Shifting evaluation from holistic semantic matching to rigorous atomic auditing, PerceptionRubrics pairs 1,038 information-dense images with over 12,000 instance-specific rubrics. These criteria are derived from golden captions constructed via a novel Circular Peer-Review consensus pipeline and then distilled into a dual-stream system of Must-Right (essential facts) and Easy-Wrong (fine-grained details) rubrics. Crucially, PerceptionRubrics implements a Gated Scoring mechanism: unlike linear averages, failure on mandatory visual facts triggers sharp binary penalties. Extensive evaluation yields critical insights: (1) The Reliability Gap: models often verify fragmented elements correctly yet fail strict conjunctive constraints, exposing brittleness in dense domains; (2) Open-Closed Stratification: contrary to reasoning trends, we reveal a persistent 8% perception deficit between open-source and proprietary frontiers; and (3) Human-Aligned Rigor: our gated metrics substantially out-align conventional benchmarks, validating that strict perceptual fidelity is the prerequisite for reliable generation.