MACS：基于质量条件的3D手部和物体运动合成

摘要

物体的物理特性，比如质量，显著影响我们用手操作物体的方式。令人惊讶的是，这一方面迄今在先前关于3D运动合成的研究中被忽视了。为了提高合成的3D手部物体运动的自然性，本研究提出了MACS，即第一个基于质量条件的3D手部和物体运动合成方法。我们的方法基于级联扩散模型，并生成根据物体质量和交互类型合理调整的交互。MACS还接受手动绘制的3D物体轨迹作为输入，并合成根据物体质量条件的自然3D手部运动。这种灵活性使得MACS可用于各种下游应用，比如为机器学习任务生成合成训练数据，用于图形工作流程中快速动画手部，以及为电脑游戏生成角色交互。我们实验证明，一个小规模数据集足以使MACS在训练期间未见过的插值和外推物体质量上合理泛化。此外，由我们的表面接触合成模型ConNet生成的质量条件接触标签使MACS对未见过的物体有适度的泛化能力。我们的全面用户研究证实，合成的3D手部物体交互非常合理和逼真。

English

The physical properties of an object, such as mass, significantly affect how we manipulate it with our hands. Surprisingly, this aspect has so far been neglected in prior work on 3D motion synthesis. To improve the naturalness of the synthesized 3D hand object motions, this work proposes MACS the first MAss Conditioned 3D hand and object motion Synthesis approach. Our approach is based on cascaded diffusion models and generates interactions that plausibly adjust based on the object mass and interaction type. MACS also accepts a manually drawn 3D object trajectory as input and synthesizes the natural 3D hand motions conditioned by the object mass. This flexibility enables MACS to be used for various downstream applications, such as generating synthetic training data for ML tasks, fast animation of hands for graphics workflows, and generating character interactions for computer games. We show experimentally that a small-scale dataset is sufficient for MACS to reasonably generalize across interpolated and extrapolated object masses unseen during the training. Furthermore, MACS shows moderate generalization to unseen objects, thanks to the mass-conditioned contact labels generated by our surface contact synthesis model ConNet. Our comprehensive user study confirms that the synthesized 3D hand-object interactions are highly plausible and realistic.

MACS：基于质量条件的3D手部和物体运动合成

MACS: Mass Conditioned 3D Hand and Object Motion Synthesis

摘要

Support