Motion4Motion: 推理时跨主体的动作迁移

摘要

本文探索了将一段视频中的运动迁移至另一段视频的技术，这对于动画中不同角色的动作生成至关重要。此前，视频运动迁移主要围绕人类及类人角色展开研究，推动了数字创作领域的诸多应用。然而，这些方法存在一个核心局限：相关技术流程高度依赖预定义的人体骨骼结构，并要求基于骨骼条件进行模型训练。一方面，这些方法难以泛化到不同物种的动物等多样角色，同时保留其独特的运动风格；另一方面，多样化骨骼的标注数据稀缺，进一步限制了该任务的大规模训练。在本文中，我们跳出基于骨骼的运动迁移框架，提出了一种无需训练的运动迁移框架——Motion4Motion。该框架通过建模视频中角色的运动流而非骨骼结构，从而更易于实现跨物种的运动迁移。大量实验与新颖应用表明，我们的方法显著优于基线模型。项目页面请见 https://lhchen.top/Motion4Motion。

English

This work explores the motion transfer from one video to another, which is crucial in animation for diverse characters. Previously, video motion transfer has been largely explored between human and human-like characters, enabling a lot of applications in digital creation. However, these approaches encounter a main limitation. Specifically, related technical pipelines heavily rely on a predefined human skeleton structure and accordingly require skeleton-conditional model training. On the one hand, these methods are difficult to generalize to diverse characters, such as animals from different species, while preserving their unique motion styles. On the other hand, labeled data in diverse skeletons is limited, which additionally restricts the large-scale training for the task. In this paper, we jump out of the skeleton-based motion transfer framework and propose a training-free motion transfer framework, named Motion4Motion. Motion4Motionmodels the motion flow of the character in a video instead of skeletons, which makes motion transfer across species easier. Extensive experimental results and novel applications show our methods outperform baselines impressively. Project page is available at https://lhchen.top/Motion4Motion.