Physics3D：通过视频扩散学习3D高斯物理特性

Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion

June 6, 2024

作者: Fangfu Liu, Hanyang Wang, Shunyu Yao, Shengjun Zhang, Jie Zhou, Yueqi Duan

cs.AI

摘要

近年来，3D生成模型取得了快速发展，为模拟3D物体的动态运动和定制其行为等应用开辟了新的可能性。然而，当前的3D生成模型往往只关注表面特征，如颜色和形状，忽视了控制物体在现实世界中行为的固有物理特性。为了准确模拟与物理一致的动态，必须预测材料的物理特性并将其纳入行为预测过程中。然而，由于现实世界物体的多样化材料具有复杂的物理属性，因此预测其物理属性仍然具有挑战性。在本文中，我们提出了Physics3D，一种通过视频扩散模型学习3D物体各种物理属性的新方法。我们的方法涉及设计基于粘弹性材料模型的高度通用的物理模拟系统，使我们能够以高保真度模拟各种材料。此外，我们从包含更多对现实物体材料理解的视频扩散模型中提取物理先验知识。大量实验证明了我们的方法在弹性和塑性材料上的有效性。Physics3D展现了极大的潜力，可以弥合物理世界与虚拟神经空间之间的差距，提供更好地在虚拟环境中整合和应用现实物理原理的可能性。项目页面：https://liuff19.github.io/Physics3D。

English

In recent years, there has been rapid development in 3D generation models, opening up new possibilities for applications such as simulating the dynamic movements of 3D objects and customizing their behaviors. However, current 3D generative models tend to focus only on surface features such as color and shape, neglecting the inherent physical properties that govern the behavior of objects in the real world. To accurately simulate physics-aligned dynamics, it is essential to predict the physical properties of materials and incorporate them into the behavior prediction process. Nonetheless, predicting the diverse materials of real-world objects is still challenging due to the complex nature of their physical attributes. In this paper, we propose Physics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model, which enables us to simulate a wide range of materials with high-fidelity capabilities. Moreover, we distill the physical priors from a video diffusion model that contains more understanding of realistic object materials. Extensive experiments demonstrate the effectiveness of our method with both elastic and plastic materials. Physics3D shows great potential for bridging the gap between the physical world and virtual neural space, providing a better integration and application of realistic physical principles in virtual environments. Project page: https://liuff19.github.io/Physics3D.