SoMA：用於機器人軟體操作的實境到模擬神經模擬器

摘要

在機器人操作的實物到模擬轉換中，模擬受豐富交互作用影響的可變形物體仍是根本性挑戰，其動力學同時受環境效應與機器人動作驅動。現有模擬器依賴預定義物理規則或未經機器人條件控制的數據驅動動力學，限制了準確性、穩定性與泛化能力。本文提出SoMA——專為軟體操作設計的3D高斯潑濺模擬器。SoMA將可變形動力學、環境作用力與機器人關節動作耦合於統一潛在神經空間，實現端到端的實物到模擬轉換。通過在學習得到的高斯潑濺模型上建立交互作用建模，該方法無需預定義物理模型即可實現可控、穩定的長時程操作，並能泛化至未觀測軌跡之外。SoMA將真實世界機器人操作的重新模擬準確度與泛化能力提升20%，可穩定模擬如長時程布料摺疊等複雜任務。

English

Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding.

SoMA：用於機器人軟體操作的實境到模擬神經模擬器

SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation

摘要

Support