VisPhyWorld：通过代码驱动视频重建探究物理推理能力

VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction

February 9, 2026

作者: Jiarong Liang, Max Ku, Ka-Hei Hui, Ping Nie, Wenhu Chen

cs.AI

摘要

评估多模态大语言模型是否真正具备物理动态推理能力仍具挑战。现有基准多采用视觉问答和预期违背等识别式评估范式，这类方法常可在无需明确物理假设的情况下作答。我们提出VisPhyWorld——一个基于执行的评估框架，通过要求模型根据视觉观察生成可执行的模拟器代码来评估物理推理能力。通过生成可运行代码，模型推断的世界表征可直接被检验、编辑和证伪，从而实现物理推理与渲染的分离。基于此框架，我们构建了包含108个物理模板衍生的209个评估场景的VisPhyBench，并制定系统化评估方案，检验模型重建外观与生成物理合理运动的能力。该流程在基准测试中实现了97.7%的有效重建视频生成率。实验表明，尽管前沿多模态大语言模型具备较强的语义场景理解能力，但在精确推断物理参数和模拟一致物理动态方面仍存在困难。

English

Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks rely on recognition-style protocols such as Visual Question Answering (VQA) and Violation of Expectation (VoE), which can often be answered without committing to an explicit, testable physical hypothesis. We propose VisPhyWorld, an execution-based framework that evaluates physical reasoning by requiring models to generate executable simulator code from visual observations. By producing runnable code, the inferred world representation is directly inspectable, editable, and falsifiable. This separates physical reasoning from rendering. Building on this framework, we introduce VisPhyBench, comprising 209 evaluation scenes derived from 108 physical templates and a systematic protocol that evaluates how well models reconstruct appearance and reproduce physically plausible motion. Our pipeline produces valid reconstructed videos in 97.7% on the benchmark. Experiments show that while state-of-the-art MLLMs achieve strong semantic scene understanding, they struggle to accurately infer physical parameters and to simulate consistent physical dynamics.