物理問題場景圖：文字生成影片中物理合理性的細粒度評估

摘要

视频生成模型越来越擅长生成逼真的视频，但仍难以生成遵循基本物理规律的视频。更棘手的是，目前缺乏可靠且细粒度的评估方法，来定位并具体描述视频中违反物理规律的情况。为此，我们提出物理问题场景图（Physics Question Scene Graph, PQSG），一种基于分层问题结构的评估流程。PQSG 通过检查生成的视频在对象、动作以及对物理规律的遵循方面是否忠实于提示内容来进行评估。它利用视觉语言模型（VLM）生成基于图的层级化问题，并以高质量的情境示例作为引导。通过将问题表示为图结构，PQSG 在问题之间引入逻辑依赖关系，确保每个查询在上下文中都是有效的。此外，PQSG 还能提供细致的评估，指出视频的哪些特质违反了物理合理性约束。我们通过创建 FinePhyEval 数据集来验证 PQSG，该数据集包含基于物理的提示语以及来自多种前沿视频生成模型（Sora 2、Veo 3 和 Wan 2.1）生成的相应视频，每个视频都由人工在多个类别上进行标注。利用 FinePhyEval，我们衡量了 PQSG 的细粒度评分与人工判断之间的相关性，结果表明其整体相关性高于以往研究。我们还发现，在物理真实感方面，PQSG 对闭源模型的排名高于 Wan 2.1。最后，我们展示 FinePhyEval 中提供的标注还可用于子任务评估：我们对两个强大的 VLM 在生成问题和回答问题方面的能力进行了基准测试，结果表明，虽然模型能够生成类似人类的问题，但在回答问题时仍逊于人类表现。

English

Video generation models are increasingly capable of producing realistic videos, but they still struggle to generate videos that follow basic physical laws. Compounding this is a lack of reliable granular evaluation methods for localizing and specifying physical law violations in videos. We address this by introducing Physics Question Scene Graph (PQSG), a hierarchical question-based evaluation pipeline. PQSG evaluates generated videos by checking their faithfulness to a prompt across objects, actions, and adherence to physical laws using a graph-based hierarchy of questions generated by a vision-language model (VLM), guided by high-quality in-context examples. By representing questions as a graph, PQSG introduces logical dependencies within questions, ensuring that each query is contextually valid. Moreover, PQSG provides granular assessments of which qualities of the video violate physical plausibility constraints. We validate PQSG by creating FinePhyEval, a dataset with physics-based prompts and corresponding generated videos from diverse state-of-the-art video generation models (Sora 2, Veo 3, and Wan 2.1), with each video annotated across multiple categories by humans. Using FinePhyEval, we measure the correlation between PQSG's fine-grained scores and human judgments, showing higher overall correlations than prior work. We also find that PQSG ranks closed-source models higher than Wan 2.1 on physical realism. Lastly, we show that the annotations we provide in FinePhyEval can also be used for subtask evaluation: we benchmark two strong VLMs on generating and answering questions, finding that while models can create human-like questions, they still fall short of human performance in answering them.