AvatarArtist：開放領域的4D化身生成技術

AvatarArtist: Open-Domain 4D Avatarization

March 25, 2025

作者: Hongyu Liu, Xuan Wang, Ziyu Wan, Yue Ma, Jingye Chen, Yanbo Fan, Yujun Shen, Yibing Song, Qifeng Chen

cs.AI

摘要

本研究聚焦於開放領域的4D虛擬化身生成，旨在從任意風格的肖像圖像中創建4D虛擬化身。我們選取參數化三平面作為中間的4D表示，並提出了一種結合生成對抗網絡（GANs）與擴散模型的實用訓練範式。這一設計源於我們觀察到，4D GANs在無監督條件下擅長於橋接圖像與三平面，但通常難以處理多樣化的數據分佈。一個強大的2D擴散先驗模型應運而生，協助GAN將其專業能力跨域遷移。這兩種技術的協同作用，促成了多領域圖像-三平面數據集的構建，從而推動了通用4D虛擬化身生成器的開發。大量實驗表明，我們的模型AvatarArtist能夠生成高質量的4D虛擬化身，並對各種源圖像領域展現出極強的魯棒性。為促進未來研究，我們將公開代碼、數據及模型。

English

This work focuses on open-domain 4D avatarization, with the purpose of creating a 4D avatar from a portrait image in an arbitrary style. We select parametric triplanes as the intermediate 4D representation and propose a practical training paradigm that takes advantage of both generative adversarial networks (GANs) and diffusion models. Our design stems from the observation that 4D GANs excel at bridging images and triplanes without supervision yet usually face challenges in handling diverse data distributions. A robust 2D diffusion prior emerges as the solution, assisting the GAN in transferring its expertise across various domains. The synergy between these experts permits the construction of a multi-domain image-triplane dataset, which drives the development of a general 4D avatar creator. Extensive experiments suggest that our model, AvatarArtist, is capable of producing high-quality 4D avatars with strong robustness to various source image domains. The code, the data, and the models will be made publicly available to facilitate future studies..