Video-MME-Logical：一个用于视频时序逻辑推理的受控诊断基准

摘要

近年来，多模态大语言模型（MLLMs）的研究热潮引发了一个核心问题：它们能否对动态视觉证据进行推理，而不仅仅是识别单帧图像中的物体或事件？我们将这种能力称为视频时序逻辑推理，它要求模型能够随着视觉状态在帧间的演变，维持、更新并组合证据。现有的视频基准测试常将这种能力与场景复杂性、静态识别或不受控的时间变化混为一谈。为了孤立地评估这一能力，我们提出了Video-MME-Logical——一个围绕五种时序逻辑操作构建的受控基准：状态追踪、顺序计数、时间排序、动态空间性和结构组合。该基准包含25个细粒度任务类别，通过受控的对象状态、状态转换、时间依赖关系和逻辑组合生成。它通过改变时间跨度和推理复杂度，支持难度可控的最终答案评估，并通过验证模型在给出最终答案前是否恢复了所需的逻辑推理轨迹，支持中间状态诊断。对最新MLLMs的实验显示，人类与模型之间存在显著差距，尤其是随着时序逻辑复杂度的增加。基于多达50万个生成样本的监督微调虽然提升了性能，但不足以弥合推理差距，这使得Video-MME-Logical成为分析和改进MLLMs时序逻辑推理能力的可扩展测试平台。

English

Recent interest in multimodal large language models (MLLMs) raises a central question: can they reason over dynamic visual evidence rather than merely recognize objects or events in individual frames? This ability, which we refer to as video temporal-logical reasoning, requires models to maintain, update, and compose evidence as visual states evolve across frames. Existing video benchmarks often conflate this capability with scene complexity, static recognition, or uncontrolled temporal variation. To isolate this capability, we introduce Video-MME-Logical, a controlled benchmark organized around five temporal-logical operations: state tracking, sequential counting, temporal ordering, dynamic spatiality, and structural composition. The benchmark contains 25 fine-grained task categories generated with controlled object states, transitions, temporal dependencies, and logical compositions. It enables difficulty-controlled final-answer evaluation by varying temporal horizon and reasoning complexity, and supports intermediate-state diagnostics by verifying whether models recover the required logical reasoning trace before producing the final answer. Experiments with state-of-the-art MLLMs reveal a substantial human-model gap, especially as temporal-logical complexity increases. Supervised fine-tuning on up to 500K generated samples improves performance but remains insufficient to close the reasoning gap, positioning Video-MME-Logical as a scalable testbed for analyzing and improving temporal-logical reasoning in MLLMs.