Animate-X：具有增強運動表示的通用角色圖像動畫

Animate-X: Universal Character Image Animation with Enhanced Motion Representation

October 14, 2024

作者: Shuai Tan, Biao Gong, Xiang Wang, Shiwei Zhang, Dandan Zheng, Ruobing Zheng, Kecheng Zheng, Jingdong Chen, Ming Yang

cs.AI

摘要

角色圖像動畫從參考圖像和目標姿勢序列生成高質量視頻，在近年來取得了顯著進展。然而，大多數現有方法僅適用於人物形象，通常無法很好地泛化應用於遊戲和娛樂等行業中常用的拟人角色。我們的深入分析表明，這種限制歸因於它們對運動建模的不足，無法理解驅動視頻的運動模式，因此將一個姿勢序列僵硬地施加在目標角色上。為此，本文提出了一種基於LDM的通用動畫框架Aniamte-X，適用於各種角色類型（統稱為X），包括拟人角色。為了增強運動表示，我們引入了姿勢指示器，通過隱式和顯式方式從驅動視頻中捕獲全面的運動模式。前者利用驅動視頻的CLIP視覺特徵提取其運動要義，如整體運動模式和運動之間的時間關係，後者通過預先模擬可能在推斷過程中出現的輸入，加強了LDM的泛化能力。此外，我們引入了一個新的動畫拟人基準（A^2Bench）來評估Animate-X在通用和廣泛應用的動畫圖像上的性能。大量實驗證明了Animate-X相對於最先進方法的優越性和有效性。

English

Character image animation, which generates high-quality videos from a reference image and target pose sequence, has seen significant progress in recent years. However, most existing methods only apply to human figures, which usually do not generalize well on anthropomorphic characters commonly used in industries like gaming and entertainment. Our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character. To this end, this paper proposes Animate-X, a universal animation framework based on LDM for various character types (collectively named X), including anthropomorphic characters. To enhance motion representation, we introduce the Pose Indicator, which captures comprehensive motion pattern from the driving video through both implicit and explicit manner. The former leverages CLIP visual features of a driving video to extract its gist of motion, like the overall movement pattern and temporal relations among motions, while the latter strengthens the generalization of LDM by simulating possible inputs in advance that may arise during inference. Moreover, we introduce a new Animated Anthropomorphic Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and widely applicable animation images. Extensive experiments demonstrate the superiority and effectiveness of Animate-X compared to state-of-the-art methods.