PhysX：基于物理的3D资产生成

PhysX: Physical-Grounded 3D Asset Generation

July 16, 2025

作者: Ziang Cao, Zhaoxi Chen, Linag Pan, Ziwei Liu

cs.AI

摘要

三维建模正从虚拟走向实体。现有的三维生成技术主要关注几何形状与纹理，却忽视了基于物理的建模。因此，尽管三维生成模型发展迅速，合成的三维资产往往忽略了丰富且关键的物理属性，这阻碍了它们在仿真、具身AI等物理领域的实际应用。作为应对这一挑战的初步尝试，我们提出了PhysX，一种端到端的基于物理的三维资产生成范式。1) 为了填补物理标注三维数据集的关键空白，我们推出了PhysXNet——首个系统性地在五个基础维度（绝对尺度、材质、功能属性、运动学及功能描述）上进行物理标注的三维数据集。特别地，我们设计了一个基于视觉-语言模型的可扩展人机协作标注流程，能够高效地将原始三维资产转化为物理优先的资产。2) 此外，我们提出了PhysXGen，一个前馈式的基于物理的图像到三维资产生成框架，它将物理知识注入预训练的三维结构空间中。具体而言，PhysXGen采用双分支架构，显式地建模三维结构与物理属性之间的潜在关联，从而在保持原有几何质量的同时，生成具有合理物理预测的三维资产。大量实验验证了我们框架的卓越性能和广阔泛化能力。所有代码、数据及模型将公开发布，以促进生成式物理AI的未来研究。

English

3D modeling is moving from virtual to physical. Existing 3D generation primarily emphasizes geometries and textures while neglecting physical-grounded modeling. Consequently, despite the rapid development of 3D generative models, the synthesized 3D assets often overlook rich and important physical properties, hampering their real-world application in physical domains like simulation and embodied AI. As an initial attempt to address this challenge, we propose PhysX, an end-to-end paradigm for physical-grounded 3D asset generation. 1) To bridge the critical gap in physics-annotated 3D datasets, we present PhysXNet - the first physics-grounded 3D dataset systematically annotated across five foundational dimensions: absolute scale, material, affordance, kinematics, and function description. In particular, we devise a scalable human-in-the-loop annotation pipeline based on vision-language models, which enables efficient creation of physics-first assets from raw 3D assets.2) Furthermore, we propose PhysXGen, a feed-forward framework for physics-grounded image-to-3D asset generation, injecting physical knowledge into the pre-trained 3D structural space. Specifically, PhysXGen employs a dual-branch architecture to explicitly model the latent correlations between 3D structures and physical properties, thereby producing 3D assets with plausible physical predictions while preserving the native geometry quality. Extensive experiments validate the superior performance and promising generalization capability of our framework. All the code, data, and models will be released to facilitate future research in generative physical AI.