PhysiFormer：学习在世界空间中模拟力学

摘要

我们提出了PhysiFormer，这是一种用于物理可信的3D物体运动的扩散变换器。与在视角相关的像素空间中操作的视频世界模型不同，PhysiFormer将物体表示为以世界坐标表达的3D网格。给定初始顶点位置和速度，以及物体材质类型（刚体或弹性体），模型采样未来的顶点轨迹。虽然相关的神经物理方法基于临时设计的潜在空间或显式强制刚性和因果性，但PhysiFormer表明，通过将顶点轨迹预测直接作为世界坐标中的单一去噪扩散过程，无需任何此类归纳偏置即可获得出色的结果。该概率公式捕捉了所学动力学中的不确定性，使得从初始条件出发能够生成多样化的合理未来，使该框架可能适用于存在未观测到不确定性的应用。模型的特征是注意力在时间、空间和物体上分解以提高效率，无需显式的物体编码即可实现排列不变的多物体推理。在超过10万个模拟轨迹上训练后，PhysiFormer能够生成刚体和弹性力学，并推广到混合材质设置、未见过的真实世界几何形状以及更多物体数量。在轨迹精度、刚性保持和基于动量的物理一致性方面，它大幅优于自回归基线模型。我们的结果将坐标空间扩散定位为通往视角不变、几何感知的世界建模的有希望的步骤，适用于机器人、图形学和物理设计。可视化、代码和模型可在https://yimingc9.github.io/physiformer获取。

English

We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.