WalkTheDog：透過相位流形的跨形態運動對齊

WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds

July 11, 2024

作者: Peizhuo Li, Sebastian Starke, Yuting Ye, Olga Sorkine-Hornung

cs.AI

摘要

我們提出了一種新方法，用於獨立於角色形態和骨骼結構的運動數據集的周期性結構和語義的理解。與現有方法使用過於稀疏的高維潛在不同，我們提出了一個包含多個閉合曲線的相位流形，每個曲線對應一個潛在振幅。通過我們提出的向量量化周期自編碼器，我們學習了一個共享的相位流形，適用於多個角色，如人類和狗，而無需任何監督。這是通過利用離散結構和淺網絡作為瓶頸來實現的，從而將語義相似的運動聚類到流形的同一曲線中，並且同一組件內的運動通過相位變量在時間上對齊。結合改進的運動匹配框架，我們展示了流形在幾個應用中的時間和語義對齊能力，包括運動檢索、轉移和風格化。本文的代碼和預先訓練模型可在 https://peizhuoli.github.io/walkthedog 上獲得。

English

We present a new approach for understanding the periodicity structure and semantics of motion datasets, independently of the morphology and skeletal structure of characters. Unlike existing methods using an overly sparse high-dimensional latent, we propose a phase manifold consisting of multiple closed curves, each corresponding to a latent amplitude. With our proposed vector quantized periodic autoencoder, we learn a shared phase manifold for multiple characters, such as a human and a dog, without any supervision. This is achieved by exploiting the discrete structure and a shallow network as bottlenecks, such that semantically similar motions are clustered into the same curve of the manifold, and the motions within the same component are aligned temporally by the phase variable. In combination with an improved motion matching framework, we demonstrate the manifold's capability of timing and semantics alignment in several applications, including motion retrieval, transfer and stylization. Code and pre-trained models for this paper are available at https://peizhuoli.github.io/walkthedog.