FantasyPortrait：通过表情增强扩散变换器提升多角色肖像动画

FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers

July 17, 2025

作者: Qiang Wang, Mengchao Wang, Fan Jiang, Yaqi Fan, Yonggang Qi, Mu Xu

cs.AI

摘要

从静态图像生成富有表现力的面部动画是一项具有挑战性的任务。以往依赖显式几何先验（如面部关键点或3DMM）的方法，在跨角色重现时常常出现伪影，且难以捕捉细微的情感变化。此外，现有方法缺乏对多角色动画的支持，因为来自不同个体的驱动特征经常相互干扰，增加了任务难度。为解决这些挑战，我们提出了FantasyPortrait，一个基于扩散变换器的框架，能够为单角色和多角色场景生成高保真且情感丰富的动画。我们的方法引入了一种表情增强学习策略，利用隐式表示来捕捉与身份无关的面部动态，从而提升模型渲染细腻情感的能力。针对多角色控制，我们设计了一种掩码交叉注意力机制，确保独立而协调的表情生成，有效防止特征干扰。为推进该领域的研究，我们提出了Multi-Expr数据集和ExprBench，这是专门为训练和评估多角色肖像动画而设计的数据集和基准。大量实验表明，FantasyPortrait在定量指标和定性评估上均显著优于现有最先进方法，尤其在具有挑战性的跨角色重现和多角色情境中表现尤为突出。我们的项目页面为https://fantasy-amap.github.io/fantasy-portrait/。

English

Producing expressive facial animations from static images is a challenging task. Prior methods relying on explicit geometric priors (e.g., facial landmarks or 3DMM) often suffer from artifacts in cross reenactment and struggle to capture subtle emotions. Furthermore, existing approaches lack support for multi-character animation, as driving features from different individuals frequently interfere with one another, complicating the task. To address these challenges, we propose FantasyPortrait, a diffusion transformer based framework capable of generating high-fidelity and emotion-rich animations for both single- and multi-character scenarios. Our method introduces an expression-augmented learning strategy that utilizes implicit representations to capture identity-agnostic facial dynamics, enhancing the model's ability to render fine-grained emotions. For multi-character control, we design a masked cross-attention mechanism that ensures independent yet coordinated expression generation, effectively preventing feature interference. To advance research in this area, we propose the Multi-Expr dataset and ExprBench, which are specifically designed datasets and benchmarks for training and evaluating multi-character portrait animations. Extensive experiments demonstrate that FantasyPortrait significantly outperforms state-of-the-art methods in both quantitative metrics and qualitative evaluations, excelling particularly in challenging cross reenactment and multi-character contexts. Our project page is https://fantasy-amap.github.io/fantasy-portrait/.