Video-MME-Logical：一個用於影片時序邏輯推理的受控診斷基準

摘要

近期對多模態大型語言模型（MLLMs）的關注引發了一個核心問題：它們能否對動態視覺證據進行推理，而不僅僅是識別單一幀中的物體或事件？這種能力，我們稱之為視頻時序邏輯推理，要求模型在視覺狀態隨幀而演變的過程中，維持、更新並整合證據。現有的視頻基準常將此能力與場景複雜性、靜態識別或不受控的時間變化混為一談。為隔離此能力，我們引入了Video-MME-Logical，一個圍繞五種時序邏輯操作（狀態追蹤、順序計數、時序排序、動態空間性與結構組合）組織的受控基準。該基準包含25個細粒度任務類別，透過受控的物件狀態、轉換、時間依賴關係與邏輯組合生成。它藉由改變時間跨度與推理複雜度，實現難度可控的最終答案評估，並透過驗證模型在產生最終答案前是否恢復所需的邏輯推理軌跡，來支持中間狀態診斷。對現有最先進MLLMs的實驗揭示了顯著的人機差距，尤其在時序邏輯複雜度增加時。在最多50萬個生成樣本上進行監督式微調雖能提升效能，但仍不足以彌合推理差距，從而使Video-MME-Logical成為一個用於分析與改進MLLMs中時序邏輯推理的可擴展測試平台。

English

Recent interest in multimodal large language models (MLLMs) raises a central question: can they reason over dynamic visual evidence rather than merely recognize objects or events in individual frames? This ability, which we refer to as video temporal-logical reasoning, requires models to maintain, update, and compose evidence as visual states evolve across frames. Existing video benchmarks often conflate this capability with scene complexity, static recognition, or uncontrolled temporal variation. To isolate this capability, we introduce Video-MME-Logical, a controlled benchmark organized around five temporal-logical operations: state tracking, sequential counting, temporal ordering, dynamic spatiality, and structural composition. The benchmark contains 25 fine-grained task categories generated with controlled object states, transitions, temporal dependencies, and logical compositions. It enables difficulty-controlled final-answer evaluation by varying temporal horizon and reasoning complexity, and supports intermediate-state diagnostics by verifying whether models recover the required logical reasoning trace before producing the final answer. Experiments with state-of-the-art MLLMs reveal a substantial human-model gap, especially as temporal-logical complexity increases. Supervised fine-tuning on up to 500K generated samples improves performance but remains insufficient to close the reasoning gap, positioning Video-MME-Logical as a scalable testbed for analyzing and improving temporal-logical reasoning in MLLMs.