SoMA：面向机器人软体操作的真实到仿真神经模拟器

摘要

在机器人操作的真实到仿真转换中，模拟具有丰富交互作用的可变形物体仍然是一个根本性挑战，其动力学同时受环境效应与机器人动作驱动。现有模拟器依赖预定义物理规则或未考虑机器人条件控制的数据驱动动力学，限制了准确性、稳定性和泛化能力。本文提出SoMA——面向软体操作的3D高斯溅射模拟器。该框架将可变形动力学、环境作用力与机器人关节动作耦合于统一潜神经空间中，实现端到端的真实到仿真模拟。通过对学习到的高斯溅射进行交互建模，系统无需预定义物理模型即可实现可控、稳定的长时程操作，并泛化至未观测轨迹之外。SoMA将真实世界机器人操作的再模拟精度与泛化能力提升20%，可稳定模拟诸如长时程布料折叠等复杂任务。

English

Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim robot manipulation, with dynamics jointly driven by environmental effects and robot actions. Existing simulators rely on predefined physics or data-driven dynamics without robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding.

SoMA：面向机器人软体操作的真实到仿真神经模拟器

SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation

摘要

Support