PhysiFormer：在世界空間中學習模擬力學

摘要

我們提出 PhysiFormer，一種用於物理合理的三維物體運動的擴散 Transformer。不同於在視角相關的像素空間中運作的視頻世界模型，PhysiFormer 將物體表示為以世界座標表達的三維網格。給定初始頂點位置與速度，以及物體材料類型（剛性或彈性），該模型會取樣未來的頂點軌跡。雖然相關的神經物理方法依賴於特定的潛在空間或明確強制剛性與因果性，PhysiFormer 顯示，無需任何此類歸納偏置，僅透過將頂點軌跡預測視為直接在世界座標中的單一去噪擴散過程，即可獲得優異結果。這種機率性公式捕捉了學習動力學中的不確定性，能從初始條件產生多樣的合理未來，使此框架可能對存在未觀測不確定性的應用有所幫助。該模型將注意力分解為時間、空間與物體以提升效率，實現排列不變的多物體推理，無需明確的物體編碼。在超過十萬條模擬軌跡上訓練後，PhysiFormer 能生成剛性與彈性力學，並泛化至混合材料設定、未見過的真實幾何形狀以及更大的物體數量。它在軌跡精度、剛性保持與基於動量的物理一致性上大幅優於自回歸基線。我們的結果將座標空間擴散定位為邁向機器人學、圖形學與物理設計中視角不變、幾何感知世界建模的有希望一步。視覺化、程式碼與模型可在 https://yimingc9.github.io/physiformer 取得。

English

We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.