GHOST 2.0：生成式高保真一次性頭像遷移

GHOST 2.0: generative high-fidelity one shot transfer of heads

February 25, 2025

作者: Alexander Groshev, Anastasiia Iashchenko, Pavel Paramonov, Denis Dimitrov, Andrey Kuznetsov

cs.AI

摘要

儘管人臉交換任務近期在研究界引起了關注，但與之相關的頭部交換問題卻仍未被深入探討。除了膚色轉移外，頭部交換還面臨額外的挑戰，例如在合成過程中需保留整個頭部的結構信息，以及修補交換頭部與背景之間的縫隙。本文中，我們通過GHOST 2.0來應對這些問題，該系統包含兩個針對特定問題的模組。首先，我們引入了增強版的對齊模型（Aligner model）用於頭部重現，該模型能在多尺度上保留身份信息，並對極端姿態變化具有魯棒性。其次，我們採用了一個混合模組（Blender module），該模組通過轉移膚色和修補不匹配區域，將重現的頭部無縫整合到目標背景中。這兩個模組在各自任務上均超越了基準模型，從而實現了頭部交換領域的頂尖成果。我們還處理了諸如源頭與目標髮型差異顯著等複雜情況。相關代碼已發佈於https://github.com/ai-forever/ghost-2.0。

English

While the task of face swapping has recently gained attention in the research community, a related problem of head swapping remains largely unexplored. In addition to skin color transfer, head swap poses extra challenges, such as the need to preserve structural information of the whole head during synthesis and inpaint gaps between swapped head and background. In this paper, we address these concerns with GHOST 2.0, which consists of two problem-specific modules. First, we introduce enhanced Aligner model for head reenactment, which preserves identity information at multiple scales and is robust to extreme pose variations. Secondly, we use a Blender module that seamlessly integrates the reenacted head into the target background by transferring skin color and inpainting mismatched regions. Both modules outperform the baselines on the corresponding tasks, allowing to achieve state of the art results in head swapping. We also tackle complex cases, such as large difference in hair styles of source and target. Code is available at https://github.com/ai-forever/ghost-2.0