基於物理的運動重新定位，從稀疏輸入開始

Physics-based Motion Retargeting from Sparse Inputs

July 4, 2023

作者: Daniele Reda, Jungdam Won, Yuting Ye, Michiel van de Panne, Alexander Winkler

cs.AI

摘要

頭像在虛擬世界中創造互動和身臨其境的體驗中扮演著重要角色。在將這些角色動畫化以模仿使用者動作方面的一個挑戰是，商用AR/VR產品僅包括頭戴式設備和控制器，提供非常有限的使用者姿勢感應數據。另一個挑戰是，頭像可能具有不同於人類的骨架結構，它們之間的映射不明確。在這項研究中，我們解決了這兩個挑戰。我們引入了一種方法，可以即時從稀疏的人體感應數據將動作重定向到不同形態的角色。我們的方法使用強化學習來訓練一個策略，以控制物理模擬器中的角色。我們只需要人體動作捕捉數據進行訓練，而無需依賴於為每個頭像生成的動畫。這使我們能夠使用大規模的動作捕捉數據集來訓練通用策略，以即時跟踪來自真實且稀疏數據的未見過的使用者。我們展示了我們的方法在具有不同骨架結構的三個角色上的可行性：恐龍、類鼠生物和人類。我們展示了頭像的姿勢通常與使用者非常契合，盡管下半身沒有可用的感應信息。我們討論並消融了我們框架中的重要組件，特別是動力學重定向步驟、模仿、接觸和動作獎勵以及我們的非對稱演員-評論者觀察。我們進一步探討了我們的方法在各種情景中的穩健性，包括不平衡、舞蹈和運動動作。

English

Avatars are important to create interactive and immersive experiences in virtual worlds. One challenge in animating these characters to mimic a user's motion is that commercial AR/VR products consist only of a headset and controllers, providing very limited sensor data of the user's pose. Another challenge is that an avatar might have a different skeleton structure than a human and the mapping between them is unclear. In this work we address both of these challenges. We introduce a method to retarget motions in real-time from sparse human sensor data to characters of various morphologies. Our method uses reinforcement learning to train a policy to control characters in a physics simulator. We only require human motion capture data for training, without relying on artist-generated animations for each avatar. This allows us to use large motion capture datasets to train general policies that can track unseen users from real and sparse data in real-time. We demonstrate the feasibility of our approach on three characters with different skeleton structure: a dinosaur, a mouse-like creature and a human. We show that the avatar poses often match the user surprisingly well, despite having no sensor information of the lower body available. We discuss and ablate the important components in our framework, specifically the kinematic retargeting step, the imitation, contact and action reward as well as our asymmetric actor-critic observations. We further explore the robustness of our method in a variety of settings including unbalancing, dancing and sports motions.