Motion4Motion：推論時的跨主體動作轉移

摘要

本論文探討將運動從一部影片遷移到另一部影片的技術，這對於動畫中多樣化角色的製作至關重要。過去，影片運動遷移主要集中於人類與類人角色之間，從而推動了許多數位創作應用的發展。然而，這些方法面臨一項主要限制：相關技術流程高度依賴預定義的人體骨骼結構，並因此需要基於骨骼條件的模型訓練。一方面，這類方法難以泛化至不同物種的動物等多樣化角色，同時保留其獨特的運動風格。另一方面，多樣化骨骼的標註數據有限，進一步限制了該任務的大規模訓練。本文跳出基於骨骼的運動遷移框架，提出一種免訓練的運動遷移框架，稱為 Motion4Motion。Motion4Motion 透過建模影片中角色的運動流（motion flow）而非骨骼，從而更容易實現跨物種的運動遷移。大量的實驗結果與新穎應用顯示，我們的方法在效能上明顯優於基準方法。專案頁面請見 https://lhchen.top/Motion4Motion 。

English

This work explores the motion transfer from one video to another, which is crucial in animation for diverse characters. Previously, video motion transfer has been largely explored between human and human-like characters, enabling a lot of applications in digital creation. However, these approaches encounter a main limitation. Specifically, related technical pipelines heavily rely on a predefined human skeleton structure and accordingly require skeleton-conditional model training. On the one hand, these methods are difficult to generalize to diverse characters, such as animals from different species, while preserving their unique motion styles. On the other hand, labeled data in diverse skeletons is limited, which additionally restricts the large-scale training for the task. In this paper, we jump out of the skeleton-based motion transfer framework and propose a training-free motion transfer framework, named Motion4Motion. Motion4Motionmodels the motion flow of the character in a video instead of skeletons, which makes motion transfer across species easier. Extensive experimental results and novel applications show our methods outperform baselines impressively. Project page is available at https://lhchen.top/Motion4Motion.