VideoCoCo:基于智能双引擎系统的代码即思维链物理一致视频生成
VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System
July 29, 2026
作者: Haodong Li, Tianfei Ren, Xiaoxiao Ma, Chunmei Qing, Zhen Fang, Sipeng He, Ziyu Guo, Haoyu Wu, Juanxi Tian, Yihang Zou, Ruichuan An, Dongzhi Jiang, Boxue Yang, Ji Xie, Xu Huang, Wenhao Yan, Jialv Zou, Zhengrong Yue, Yaxin Luo, Xiaotong Li, Yuzhu Wang, Junyan Ye, Jinjing Zhao, Zehui Chen, Lin Chen, Renye Yan, Feng Zhao, Pheng-Ann Heng
cs.AI
摘要
文生视频模型已取得了卓越的视觉质量,但由于场景的时间演化必须从高度压缩的文本提示中隐式推断,它们仍然难以生成物理一致的动力学。现有的思维链方法引入了中间计划或视觉状态,但这些表示通常不可执行或在时间上稀疏,限制了它们实例化和控制完整时空过程的能力。为解决这一局限性,我们提出了 VideoCoCo,一种智能体双引擎框架,其中可执行的 Blender 代码充当过程级思维链。给定文本提示,代码智能体生成一个 Blender 程序,该程序明确指定场景及其时间演化。可执行仿真引擎运行该程序以产生确定性的时空草稿,随后通过基于草稿条件的编辑,由生成式视频引擎将其转换为照片级真实感视频。这种分解将过程级推理与高保真视觉实现分离开来。为了使视频编辑器适配仿真草稿,我们构建了 VideoCoCo-3K,一个包含草稿-指令-目标三元组的精选数据集。VideoCoCo 将 OmniWeaving 基线在 PhyGenBench 上从 0.475 提升至 0.558,在 VBench-2.0 上从 52.18 提升至 77.88,在两个基准上均取得了最佳平均分数。这些结果表明,可执行代码为物理一致的视频生成提供了一种有效、可控且可检查的中间表示。
English
Text-to-video models have achieved remarkable visual quality, yet they still struggle to generate physically consistent dynamics because the temporal evolution of a scene must be inferred implicitly from a highly compressed text prompt. Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-executable or temporally sparse, limiting their ability to instantiate and control the complete spatiotemporal process. To address this limitation, we introduce VideoCoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. The executable simulation engine runs the program to produce a deterministic spatiotemporal draft, which is subsequently transformed into a photorealistic video by a generative video engine through draft-conditioned editing. This decomposition separates process-level reasoning from high-fidelity visual realization. To adapt the video editor to simulated drafts, we construct VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.