WalkTheDog：通过相位流形实现跨形态运动对齐

WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds

July 11, 2024

作者: Peizhuo Li, Sebastian Starke, Yuting Ye, Olga Sorkine-Hornung

cs.AI

摘要

我们提出了一种新方法，用于独立于角色形态和骨骼结构的情况下理解运动数据集的周期性结构和语义。与现有方法使用过于稀疏的高维潜变量不同，我们提出了一个包含多个闭合曲线的相位流形，每个曲线对应一个潜在振幅。通过我们提出的矢量量化周期自动编码器，我们学习了一个共享的相位流形，适用于多个角色，如人类和狗，而无需任何监督。这是通过利用离散结构和浅层网络作为瓶颈来实现的，从而使语义上相似的运动被聚类到流形的同一曲线中，并且同一组件内的运动通过相位变量在时间上对齐。结合改进的运动匹配框架，我们展示了该流形在多个应用中进行时间和语义对齐的能力，包括运动检索、转移和风格化。本文的代码和预训练模型可在https://peizhuoli.github.io/walkthedog 上找到。

English

We present a new approach for understanding the periodicity structure and semantics of motion datasets, independently of the morphology and skeletal structure of characters. Unlike existing methods using an overly sparse high-dimensional latent, we propose a phase manifold consisting of multiple closed curves, each corresponding to a latent amplitude. With our proposed vector quantized periodic autoencoder, we learn a shared phase manifold for multiple characters, such as a human and a dog, without any supervision. This is achieved by exploiting the discrete structure and a shallow network as bottlenecks, such that semantically similar motions are clustered into the same curve of the manifold, and the motions within the same component are aligned temporally by the phase variable. In combination with an improved motion matching framework, we demonstrate the manifold's capability of timing and semantics alignment in several applications, including motion retrieval, transfer and stylization. Code and pre-trained models for this paper are available at https://peizhuoli.github.io/walkthedog.