VGenST-Bench：通过主动视频合成进行时空推理的基准测试

摘要

时空推理是现实世界中多模态大语言模型（MLLMs）的核心能力之一。因此，对其进行精确评估已成为一项关键挑战。然而，现有的时空推理基准数据集主要依赖静态图像集或被动策划的视频数据，这限制了细粒度推理能力的评估。本文提出VGenST-Bench——一个利用生成模型主动合成高度可控且多样化的评估场景的视频基准。为构建VGenST-Bench，我们设计了一个多智能体流水线，其中包含人工质量控制环节，以确保所有生成视频及问答对的质量。我们建立了一个全面的3×2×2视频分类体系，涵盖空间尺度、视角和场景动态，以覆盖多样化的场景。此外，我们设计了一套层次化任务套件，将低层级视觉感知与高层级时空推理相解耦。通过将范式从被动策划转向主动合成，VGenST-Bench能够实现对MLLMs时空理解能力的细粒度诊断。

English

Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-temporal reasoning benchmark datasets primarily rely on static image sets or passively curated video data, which limits the evaluation of fine-grained reasoning capabilities. In this paper, we introduce VGenST-Bench, a video benchmark that employs generative models to actively synthesize highly controlled and diverse evaluation scenarios. To construct VGenST-Bench, we propose a multi-agent pipeline incorporating a human quality control stage, ensuring the quality of all generated videos and QA pairs. We establish a comprehensive 3x2x2 video taxonomy, encompassing Spatial Scale, Perspective, and Scene Dynamics to span diverse scenarios. Furthermore, we design a hierarchical task suite that decouples low-level visual perception from high-level spatio-temporal reasoning. By shifting the paradigm from passive curation to active synthesis, VGenST-Bench enables fine-grained diagnosis of spatio-temporal understanding in MLLMs.