MMPerspective：多模态大语言模型能否理解视角？一个全面的视角感知、推理与鲁棒性基准

MMPerspective: Do MLLMs Understand Perspective? A Comprehensive Benchmark for Perspective Perception, Reasoning, and Robustness

May 26, 2025

作者: Yunlong Tang, Pinxin Liu, Mingqian Feng, Zhangyun Tan, Rui Mao, Chao Huang, Jing Bi, Yunzhong Xiao, Susan Liang, Hang Hua, Ali Vosoughi, Luchuan Song, Zeliang Zhang, Chenliang Xu

cs.AI

摘要

理解透视是人类视觉感知的基础，然而多模态大语言模型（MLLMs）在多大程度上内化了透视几何仍不明确。我们推出了MMPerspective，这是首个专门设计用于系统评估MLLMs透视理解的基准，通过三个互补维度——透视感知、推理与鲁棒性——下的10项精心设计的任务来实现。该基准包含2,711个真实世界与合成图像实例，以及5,083个问答对，旨在探测关键能力，如消失点感知与计数、透视类型推理、三维空间中的线条关系理解、对透视保持变换的不变性等。通过对43个顶尖MLLMs的全面评估，我们揭示了显著的局限性：尽管模型在表面层次的感知任务上表现出色，但在组合推理及受扰动时保持空间一致性方面却面临挑战。我们的分析进一步揭示了模型架构、规模与透视能力之间的有趣关联，既指出了鲁棒性瓶颈，也展示了链式思维提示的益处。MMPerspective为诊断和推进视觉语言系统中的空间理解建立了一个宝贵的测试平台。资源获取地址：https://yunlong10.github.io/MMPerspective/

English

Understanding perspective is fundamental to human visual perception, yet the extent to which multimodal large language models (MLLMs) internalize perspective geometry remains unclear. We introduce MMPerspective, the first benchmark specifically designed to systematically evaluate MLLMs' understanding of perspective through 10 carefully crafted tasks across three complementary dimensions: Perspective Perception, Reasoning, and Robustness. Our benchmark comprises 2,711 real-world and synthetic image instances with 5,083 question-answer pairs that probe key capabilities, such as vanishing point perception and counting, perspective type reasoning, line relationship understanding in 3D space, invariance to perspective-preserving transformations, etc. Through a comprehensive evaluation of 43 state-of-the-art MLLMs, we uncover significant limitations: while models demonstrate competence on surface-level perceptual tasks, they struggle with compositional reasoning and maintaining spatial consistency under perturbations. Our analysis further reveals intriguing patterns between model architecture, scale, and perspective capabilities, highlighting both robustness bottlenecks and the benefits of chain-of-thought prompting. MMPerspective establishes a valuable testbed for diagnosing and advancing spatial understanding in vision-language systems. Resources available at: https://yunlong10.github.io/MMPerspective/