VGenST-Bench：透過主動影片合成進行時空推理的基準

摘要

時空推理是運行於現實世界中的多模態大型語言模型（MLLMs）一項核心能力。因此，精確評估此能力已成為一項關鍵挑戰。然而，現有的時空推理基準資料集主要依賴靜態影像集或被動彙整的影片資料，限制了對細粒度推理能力的評估。在本文中，我們提出 VGenST-Bench，這是一個利用生成模型主動合成高度可控且多樣化評估場景的影片基準。為建構 VGenST-Bench，我們提出一個包含人工品質控制階段的多智能體流程，確保所有生成影片及問答對的品質。我們建立了一個全面的 3×2×2 影片分類體系，涵蓋空間尺度、視角與場景動態，以覆蓋多樣化場景。此外，我們設計了一套層次化任務套件，將低層次視覺感知與高層次時空推理分離開來。透過將範式從被動彙整轉變為主動合成，VGenST-Bench 得以對 MLLMs 的時空理解進行細粒度診斷。

English

Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-temporal reasoning benchmark datasets primarily rely on static image sets or passively curated video data, which limits the evaluation of fine-grained reasoning capabilities. In this paper, we introduce VGenST-Bench, a video benchmark that employs generative models to actively synthesize highly controlled and diverse evaluation scenarios. To construct VGenST-Bench, we propose a multi-agent pipeline incorporating a human quality control stage, ensuring the quality of all generated videos and QA pairs. We establish a comprehensive 3x2x2 video taxonomy, encompassing Spatial Scale, Perspective, and Scene Dynamics to span diverse scenarios. Furthermore, we design a hierarchical task suite that decouples low-level visual perception from high-level spatio-temporal reasoning. By shifting the paradigm from passive curation to active synthesis, VGenST-Bench enables fine-grained diagnosis of spatio-temporal understanding in MLLMs.