合成视频提升了视频合成中的物理真实感

Synthetic Video Enhances Physical Fidelity in Video Synthesis

March 26, 2025

作者: Qi Zhao, Xingyu Ni, Ziyu Wang, Feng Cheng, Ziyan Yang, Lu Jiang, Bohan Wang

cs.AI

摘要

我们探索了如何通过利用计算机图形管线生成的合成视频来提升视频生成模型的物理真实感。这些渲染视频遵循现实世界的物理规律，如保持三维一致性，为改进视频生成模型提供了宝贵的资源。为了充分发挥这一潜力，我们提出了一种解决方案，既精心筛选并整合合成数据，又引入了一种方法，将合成数据的物理真实感迁移至模型中，从而显著减少不期望的伪影。通过在三个强调物理一致性的代表性任务上的实验，我们验证了该方法在提升物理真实感方面的有效性。尽管我们的模型尚未深入理解物理规律，但我们的工作首次通过实证表明，合成视频能够增强视频合成中的物理真实感。项目网站：https://kevinz8866.github.io/simulation/

English

We investigate how to enhance the physical fidelity of video generation models by leveraging synthetic videos derived from computer graphics pipelines. These rendered videos respect real-world physics, such as maintaining 3D consistency, and serve as a valuable resource that can potentially improve video generation models. To harness this potential, we propose a solution that curates and integrates synthetic data while introducing a method to transfer its physical realism to the model, significantly reducing unwanted artifacts. Through experiments on three representative tasks emphasizing physical consistency, we demonstrate its efficacy in enhancing physical fidelity. While our model still lacks a deep understanding of physics, our work offers one of the first empirical demonstrations that synthetic video enhances physical fidelity in video synthesis. Website: https://kevinz8866.github.io/simulation/