合成影片提升了影片合成中的物理真實感

Synthetic Video Enhances Physical Fidelity in Video Synthesis

March 26, 2025

作者: Qi Zhao, Xingyu Ni, Ziyu Wang, Feng Cheng, Ziyan Yang, Lu Jiang, Bohan Wang

cs.AI

摘要

我們探討如何利用源自計算機圖形管線的合成視頻來提升視頻生成模型的物理真實感。這些渲染的視頻遵循現實世界的物理規律，例如保持三維一致性，並作為一種寶貴資源，有可能改進視頻生成模型。為挖掘這一潛力，我們提出了一種解決方案，該方案精心策劃並整合合成數據，同時引入了一種方法，將這些數據的物理真實感轉移到模型中，從而顯著減少不想要的偽影。通過在三個強調物理一致性的代表性任務上的實驗，我們證明了其在增強物理真實感方面的有效性。儘管我們的模型仍缺乏對物理的深入理解，但我們的工作提供了首批實證之一，表明合成視頻能夠提升視頻合成中的物理真實感。網站：https://kevinz8866.github.io/simulation/

English

We investigate how to enhance the physical fidelity of video generation models by leveraging synthetic videos derived from computer graphics pipelines. These rendered videos respect real-world physics, such as maintaining 3D consistency, and serve as a valuable resource that can potentially improve video generation models. To harness this potential, we propose a solution that curates and integrates synthetic data while introducing a method to transfer its physical realism to the model, significantly reducing unwanted artifacts. Through experiments on three representative tasks emphasizing physical consistency, we demonstrate its efficacy in enhancing physical fidelity. While our model still lacks a deep understanding of physics, our work offers one of the first empirical demonstrations that synthetic video enhances physical fidelity in video synthesis. Website: https://kevinz8866.github.io/simulation/