PhysWorld：通过物理感知演示合成实现从真实视频到可变形物体的世界建模

摘要

能够模拟物体动态的交互式世界模型对于机器人技术、虚拟现实和增强现实至关重要。然而，从有限的真实世界视频数据中学习物理一致性动力学模型仍面临重大挑战，特别是针对具有空间变化物理属性的可变形物体。为克服数据稀缺的难题，我们提出PhysWorld——一种创新框架，通过利用模拟器合成物理合理且多样化的演示数据来学习高效的世界模型。具体而言，我们首先通过本构模型选择和物理属性全局-局部优化，在MPM模拟器中构建物理一致的数字孪生体；随后对物理属性施加部件感知扰动，为数字孪生体生成多样化运动模式，从而合成大规模异构演示数据；最后基于这些演示数据训练嵌入物理属性的轻量级图神经网络世界模型。真实视频数据可进一步用于优化物理属性。PhysWorld实现了对各类可变形物体的精准快速未来预测，并展现出良好的新交互泛化能力。实验表明，PhysWorld在保持竞争力的同时，推理速度较当前最先进方法PhysTwin提升47倍。

English

Interactive world models that simulate object dynamics are crucial for robotics, VR, and AR. However, it remains a significant challenge to learn physics-consistent dynamics models from limited real-world video data, especially for deformable objects with spatially-varying physical properties. To overcome the challenge of data scarcity, we propose PhysWorld, a novel framework that utilizes a simulator to synthesize physically plausible and diverse demonstrations to learn efficient world models. Specifically, we first construct a physics-consistent digital twin within MPM simulator via constitutive model selection and global-to-local optimization of physical properties. Subsequently, we apply part-aware perturbations to the physical properties and generate various motion patterns for the digital twin, synthesizing extensive and diverse demonstrations. Finally, using these demonstrations, we train a lightweight GNN-based world model that is embedded with physical properties. The real video can be used to further refine the physical properties. PhysWorld achieves accurate and fast future predictions for various deformable objects, and also generalizes well to novel interactions. Experiments show that PhysWorld has competitive performance while enabling inference speeds 47 times faster than the recent state-of-the-art method, i.e., PhysTwin.

PhysWorld：通过物理感知演示合成实现从真实视频到可变形物体的世界建模

PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesis

摘要

Support