X-Dyna：具表現力的動態人像動畫

摘要

我們介紹了 X-Dyna，一種新穎的零樣本、基於擴散的流程，用於通過從驅動視頻中提取的面部表情和身體動作來為單張人像圖像添加動畫效果，生成既真實又具有上下文感知的動態效果，涵蓋了主題及周圍環境。在以人體姿勢控制為中心的先前方法基礎上，X-Dyna 解決了導致動態細節丟失的主要缺陷，增強了人類視頻動畫的逼真特性。我們方法的核心是 Dynamics-Adapter，這是一個輕量級模塊，能夠有效地將參考外觀上下文整合到擴散主幹的空間關注中，同時保留運動模塊在合成流暢和複雜動態細節方面的能力。除了身體姿勢控制，我們還將本地控制模塊與我們的模型相連接，以捕獲與身份解耦的面部表情，從而實現準確的表情轉移，增強動畫場景的逼真感。這些組件共同構成了一個統一的框架，能夠從各種人類和場景視頻中學習人類運動和自然場景動態。全面的定性和定量評估表明，X-Dyna 優於最先進的方法，創建出高度逼真和富有表現力的動畫。代碼可在 https://github.com/bytedance/X-Dyna 找到。

English

We introduce X-Dyna, a novel zero-shot, diffusion-based pipeline for animating a single human image using facial expressions and body movements derived from a driving video, that generates realistic, context-aware dynamics for both the subject and the surrounding environment. Building on prior approaches centered on human pose control, X-Dyna addresses key shortcomings causing the loss of dynamic details, enhancing the lifelike qualities of human video animations. At the core of our approach is the Dynamics-Adapter, a lightweight module that effectively integrates reference appearance context into the spatial attentions of the diffusion backbone while preserving the capacity of motion modules in synthesizing fluid and intricate dynamic details. Beyond body pose control, we connect a local control module with our model to capture identity-disentangled facial expressions, facilitating accurate expression transfer for enhanced realism in animated scenes. Together, these components form a unified framework capable of learning physical human motion and natural scene dynamics from a diverse blend of human and scene videos. Comprehensive qualitative and quantitative evaluations demonstrate that X-Dyna outperforms state-of-the-art methods, creating highly lifelike and expressive animations. The code is available at https://github.com/bytedance/X-Dyna.

X-Dyna：具表現力的動態人像動畫

X-Dyna: Expressive Dynamic Human Image Animation

摘要

Summary

Support

Support