DiffPortrait360：面向360度视角合成的连贯肖像扩散模型

DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesis

March 19, 2025

作者: Yuming Gu, Phong Tran, Yujian Zheng, Hongyi Xu, Heyuan Li, Adilbek Karmanov, Hao Li

cs.AI

摘要

从单视角图像生成高质量的人头360度视图，对于实现便捷的沉浸式远程呈现应用和可扩展的个性化内容创作至关重要。尽管当前最先进的全头生成方法仅限于建模逼真的人头，而最新的基于扩散的风格全知头部合成技术仅能生成正面视图，且在视角一致性上存在困难，阻碍了其转化为可从任意角度渲染的真实3D模型。我们提出了一种创新方法，能够生成完全一致的360度头部视图，适用于人类、风格化及拟人化形态，包括眼镜和帽子等配饰。该方法基于DiffPortrait3D框架，融入了一个定制ControlNet用于后脑细节生成，以及一个双重外观模块以确保全局前后一致性。通过对连续视角序列的训练并结合后部参考图像，我们的方法实现了稳健、局部连续的视角合成。该模型可用于生成高质量的神经辐射场（NeRFs），支持实时自由视点渲染，在极具挑战性的输入肖像下，于物体合成和360度头部生成方面超越了现有最先进方法。

English

Generating high-quality 360-degree views of human heads from single-view images is essential for enabling accessible immersive telepresence applications and scalable personalized content creation. While cutting-edge methods for full head generation are limited to modeling realistic human heads, the latest diffusion-based approaches for style-omniscient head synthesis can produce only frontal views and struggle with view consistency, preventing their conversion into true 3D models for rendering from arbitrary angles. We introduce a novel approach that generates fully consistent 360-degree head views, accommodating human, stylized, and anthropomorphic forms, including accessories like glasses and hats. Our method builds on the DiffPortrait3D framework, incorporating a custom ControlNet for back-of-head detail generation and a dual appearance module to ensure global front-back consistency. By training on continuous view sequences and integrating a back reference image, our approach achieves robust, locally continuous view synthesis. Our model can be used to produce high-quality neural radiance fields (NeRFs) for real-time, free-viewpoint rendering, outperforming state-of-the-art methods in object synthesis and 360-degree head generation for very challenging input portraits.