Disco4D：从单个图像实现解耦的4D人体生成与动画

Disco4D: Disentangled 4D Human Generation and Animation from a Single Image

September 25, 2024

作者: Hui En Pang, Shuai Liu, Zhongang Cai, Lei Yang, Tianwei Zhang, Ziwei Liu

cs.AI

摘要

我们提出了Disco4D，这是一个新颖的高斯飘粒框架，用于从单个图像生成和动画化4D人物。与现有方法不同，Disco4D独特地将服装（使用高斯模型）与人体（使用SMPL-X模型）进行了解耦，显著增强了生成细节和灵活性。它具有以下技术创新。1）Disco4D学习有效地将服装高斯分布拟合到SMPL-X高斯分布上。2）它采用扩散模型来增强3D生成过程，例如，对输入图像中不可见的遮挡部分进行建模。3）它为每个服装高斯分布学习了一个身份编码，以促进服装资产的分离和提取。此外，Disco4D自然地支持具有生动动态的4D人物动画。大量实验证明了Disco4D在4D人物生成和动画任务上的优越性。我们的可视化结果可以在https://disco-4d.github.io/找到。

English

We present Disco4D, a novel Gaussian Splatting framework for 4D human generation and animation from a single image. Different from existing methods, Disco4D distinctively disentangles clothings (with Gaussian models) from the human body (with SMPL-X model), significantly enhancing the generation details and flexibility. It has the following technical innovations. 1) Disco4D learns to efficiently fit the clothing Gaussians over the SMPL-X Gaussians. 2) It adopts diffusion models to enhance the 3D generation process, e.g., modeling occluded parts not visible in the input image. 3) It learns an identity encoding for each clothing Gaussian to facilitate the separation and extraction of clothing assets. Furthermore, Disco4D naturally supports 4D human animation with vivid dynamics. Extensive experiments demonstrate the superiority of Disco4D on 4D human generation and animation tasks. Our visualizations can be found in https://disco-4d.github.io/.