MACS：質量條件下的3D手部和物體運動合成

摘要

物體的物理特性，如質量，顯著影響我們用手操作它的方式。令人驚訝的是，這個方面在先前的3D動作合成研究中迄今被忽略了。為了提高合成的3D手部物體運動的自然性，本研究提出了MACS，這是第一個基於質量條件的3D手部和物體運動合成方法。我們的方法基於級聯擴散模型，生成的互動會根據物體的質量和互動類型合理調整。MACS還接受手動繪製的3D物體軌跡作為輸入，並合成根據物體質量條件的自然3D手部運動。這種靈活性使MACS可用於各種下游應用，例如為ML任務生成合成訓練數據，快速為圖形工作流程製作手部動畫，以及為電腦遊戲生成角色互動。我們的實驗表明，一個小規模數據集就足以使MACS在訓練期間未見的插值和外插物體質量上合理泛化。此外，由我們的表面接觸合成模型ConNet生成的質量條件接觸標籤使MACS對未見的物體有中等泛化能力。我們的全面用戶研究證實了合成的3D手部物體互動是高度合理和逼真的。

English

The physical properties of an object, such as mass, significantly affect how we manipulate it with our hands. Surprisingly, this aspect has so far been neglected in prior work on 3D motion synthesis. To improve the naturalness of the synthesized 3D hand object motions, this work proposes MACS the first MAss Conditioned 3D hand and object motion Synthesis approach. Our approach is based on cascaded diffusion models and generates interactions that plausibly adjust based on the object mass and interaction type. MACS also accepts a manually drawn 3D object trajectory as input and synthesizes the natural 3D hand motions conditioned by the object mass. This flexibility enables MACS to be used for various downstream applications, such as generating synthetic training data for ML tasks, fast animation of hands for graphics workflows, and generating character interactions for computer games. We show experimentally that a small-scale dataset is sufficient for MACS to reasonably generalize across interpolated and extrapolated object masses unseen during the training. Furthermore, MACS shows moderate generalization to unseen objects, thanks to the mass-conditioned contact labels generated by our surface contact synthesis model ConNet. Our comprehensive user study confirms that the synthesized 3D hand-object interactions are highly plausible and realistic.

MACS：質量條件下的3D手部和物體運動合成

MACS: Mass Conditioned 3D Hand and Object Motion Synthesis

摘要

Support