代码即世界:智能体驱动的可执行世界表征发现,用于物理推理
Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning
August 27, 2026
作者: Hanyang Wang, Yimo Cai, Weiliang Chen, Jiawei Chi, Haowen Sun, Qiyu Dai, Yi-Hsin Hung, Xingzhuo Guo, Jinshan Ren, Runmao Yao, Ziwei Liu, Mingsheng Long, Yueqi Duan, Jun Gao, Jiangran Lyu, Fangfu Liu, Jialong Wu
cs.AI
摘要
物理理解与推理依赖于对世界形成紧凑且可泛化的表征。尽管现代视觉-语言模型能够识别并解释多样的物理事件,它们往往缺乏对底层机制(如物体状态、物理参数及控制动力学)的显式表征,而这些机制正是可靠推理世界如何演化以及如何响应干预所必需的。在本工作中,我们提出Code-as-World范式,通过可执行的世界表征来表示物理世界。通过将物理组成、动态演化与视觉外观表达为可执行代码,Code-as-World提供了对物理世界的一种紧凑、定量可计算且可控的抽象。为了从多模态观测(如自然语言描述或真实世界视频)中构建此类表征,我们受溯因推理启发,开发了一种智能体发现循环,其中智能体对可执行的世界假设进行提出、执行、渲染、验证及迭代细化。作为具体应用,我们利用经过验证的可执行世界,为视觉-语言模型在定量物理推理任务上提供可扩展的物理监督。实验表明,Code-as-World-VL在QuantiPhy基准上达到了最先进的性能,并超越了领先的专有模型,凸显了可执行世界表征作为物理智能可扩展基础的潜力。
English
Physical understanding and reasoning depend on forming compact and generalizable representations of the world. While modern vision-language models can recognize and explain diverse physical events, they often lack explicit representations of the underlying mechanisms-such as object states, physical parameters, and governing dynamics-needed for reliably reasoning how the world evolves and responds to interventions. In this work, we introduce Code-as-World, a paradigm that represents physical worlds through executable world representations. By expressing physical composition, dynamic evolution, and visual appearance as executable code, Code-as-World provides a compact, quantitatively grounded, and controllable abstraction of the physical world. To construct such representations from multimodal observations, such as natural-language descriptions or real-world videos, we develop an agentic discovery loop inspired by abductive reasoning, where an agent proposes, executes, renders, verifies, and iteratively refines executable world hypotheses. As a concrete application, we use verified executable worlds to provide scalable physical supervision for training vision-language models on quantitative physical reasoning. Experiments show that Code-as-World-VL achieves state-of-the-art performance on QuantiPhy and surpasses leading proprietary models, highlighting the potential of executable world representations as a scalable foundation for physical intelligence.