Physics3D：通過視頻擴散學習3D高斯函數的物理特性

Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion

June 6, 2024

作者: Fangfu Liu, Hanyang Wang, Shunyu Yao, Shengjun Zhang, Jie Zhou, Yueqi Duan

cs.AI

摘要

近年來，3D生成模型有了快速發展，為模擬3D物體的動態運動和自定義行為等應用開創了新的可能性。然而，目前的3D生成模型往往僅關注表面特徵，如顏色和形狀，忽略了控制現實世界中物體行為的固有物理特性。為了準確模擬與物理相符的動態，必須預測材料的物理特性並將其納入行為預測過程中。然而，由於現實世界物體的多樣材料具有複雜的物理屬性，因此預測它們仍然具有挑戰性。本文提出了Physics3D，一種通過視頻擴散模型學習3D物體各種物理特性的新方法。我們的方法涉及設計一個基於粘彈性材料模型的高度通用的物理模擬系統，使我們能夠以高保真度模擬各種材料。此外，我們從包含更多對現實物體材料理解的視頻擴散模型中提煉出物理先驗知識。大量實驗證明了我們的方法對彈性和塑性材料的有效性。Physics3D展現了將物理世界與虛擬神經空間之間的鴻溝，提供更好地在虛擬環境中整合和應用現實物理原則的潛力。項目頁面：https://liuff19.github.io/Physics3D。

English

In recent years, there has been rapid development in 3D generation models, opening up new possibilities for applications such as simulating the dynamic movements of 3D objects and customizing their behaviors. However, current 3D generative models tend to focus only on surface features such as color and shape, neglecting the inherent physical properties that govern the behavior of objects in the real world. To accurately simulate physics-aligned dynamics, it is essential to predict the physical properties of materials and incorporate them into the behavior prediction process. Nonetheless, predicting the diverse materials of real-world objects is still challenging due to the complex nature of their physical attributes. In this paper, we propose Physics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model, which enables us to simulate a wide range of materials with high-fidelity capabilities. Moreover, we distill the physical priors from a video diffusion model that contains more understanding of realistic object materials. Extensive experiments demonstrate the effectiveness of our method with both elastic and plastic materials. Physics3D shows great potential for bridging the gap between the physical world and virtual neural space, providing a better integration and application of realistic physical principles in virtual environments. Project page: https://liuff19.github.io/Physics3D.