VisPhyWorld：通过代码驱动视频重建探究物理推理能力

VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstruction

February 9, 2026

作者: Jiarong Liang, Max Ku, Ka-Hei Hui, Ping Nie, Wenhu Chen

cs.AI

摘要

评估多模态大语言模型是否真正具备物理动态推理能力仍具挑战。现有基准多采用视觉问答和期望违背等识别式范式，这类任务往往无需模型构建明确可验证的物理假设即可作答。我们提出VisPhyWorld——一个基于执行的评估框架，通过要求模型根据视觉观察生成可执行的模拟器代码来检验物理推理能力。通过生成可运行代码，模型推断的世界表征可直接被检验、编辑和证伪，从而将物理推理与渲染过程分离。基于该框架，我们构建了包含108个物理模板生成的209个评估场景的VisPhyBench，并制定系统化评估方案，检验模型重建物体外观与生成物理合理运动的能力。该流程在基准测试中实现了97.7%的有效重建视频生成率。实验表明，尽管前沿多模态大语言模型具备较强的场景语义理解能力，但在精确推断物理参数和模拟一致性物理动态方面仍存在明显不足。

English

Evaluating whether Multimodal Large Language Models (MLLMs) genuinely reason about physical dynamics remains challenging. Most existing benchmarks rely on recognition-style protocols such as Visual Question Answering (VQA) and Violation of Expectation (VoE), which can often be answered without committing to an explicit, testable physical hypothesis. We propose VisPhyWorld, an execution-based framework that evaluates physical reasoning by requiring models to generate executable simulator code from visual observations. By producing runnable code, the inferred world representation is directly inspectable, editable, and falsifiable. This separates physical reasoning from rendering. Building on this framework, we introduce VisPhyBench, comprising 209 evaluation scenes derived from 108 physical templates and a systematic protocol that evaluates how well models reconstruct appearance and reproduce physically plausible motion. Our pipeline produces valid reconstructed videos in 97.7% on the benchmark. Experiments show that while state-of-the-art MLLMs achieve strong semantic scene understanding, they struggle to accurately infer physical parameters and to simulate consistent physical dynamics.