DiffPortrait360：用於360度視角合成的統一肖像擴散模型

DiffPortrait360: Consistent Portrait Diffusion for 360 View Synthesis

March 19, 2025

作者: Yuming Gu, Phong Tran, Yujian Zheng, Hongyi Xu, Heyuan Li, Adilbek Karmanov, Hao Li

cs.AI

摘要

從單一視角圖像生成高品質的360度人頭視圖，對於實現可訪問的沉浸式遠程呈現應用和可擴展的個性化內容創作至關重要。儘管現有的全頭部生成尖端方法僅限於建模逼真的人類頭部，而最新的基於擴散技術的風格全知頭部合成方法只能生成正面視圖，且在視圖一致性方面存在困難，這阻礙了它們轉化為真正的3D模型以從任意角度渲染。我們提出了一種新穎的方法，能夠生成完全一致的360度頭部視圖，適用於人類、風格化以及擬人化形態，包括眼鏡和帽子等配飾。我們的方法基於DiffPortrait3D框架，結合了自定義的ControlNet用於後腦細節生成，以及雙重外觀模塊以確保全局前後一致性。通過在連續視圖序列上進行訓練並整合後參考圖像，我們的方法實現了穩健、局部連續的視圖合成。我們的模型可用於生成高品質的神經輻射場（NeRFs），用於實時、自由視點的渲染，在極具挑戰性的輸入肖像的物體合成和360度頭部生成方面，超越了現有最先進的方法。

English

Generating high-quality 360-degree views of human heads from single-view images is essential for enabling accessible immersive telepresence applications and scalable personalized content creation. While cutting-edge methods for full head generation are limited to modeling realistic human heads, the latest diffusion-based approaches for style-omniscient head synthesis can produce only frontal views and struggle with view consistency, preventing their conversion into true 3D models for rendering from arbitrary angles. We introduce a novel approach that generates fully consistent 360-degree head views, accommodating human, stylized, and anthropomorphic forms, including accessories like glasses and hats. Our method builds on the DiffPortrait3D framework, incorporating a custom ControlNet for back-of-head detail generation and a dual appearance module to ensure global front-back consistency. By training on continuous view sequences and integrating a back reference image, our approach achieves robust, locally continuous view synthesis. Our model can be used to produce high-quality neural radiance fields (NeRFs) for real-time, free-viewpoint rendering, outperforming state-of-the-art methods in object synthesis and 360-degree head generation for very challenging input portraits.