HumanSplat：具有結構先驗的通用單圖像人類高斯飛濺

HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors

June 18, 2024

作者: Panwang Pan, Zhuo Su, Chenguo Lin, Zhen Fan, Yongjie Zhang, Zeming Li, Tingting Shen, Yadong Mu, Yebin Liu

cs.AI

摘要

儘管最近在高保真人類重建技術方面取得了進展，但對密集捕捉圖像或耗時的每個實例優化的要求顯著阻礙了它們在更廣泛場景中的應用。為了應對這些問題，我們提出了HumanSplat，它以通用方式從單個輸入圖像預測任何人的3D高斯Splatting屬性。具體而言，HumanSplat包括一個2D多視圖擴散模型和一個具有人體結構先驗的潛在重建轉換器，巧妙地在統一框架內整合了幾何先驗和語義特徵。進一步設計了一種包含人體語義信息的分層損失，以實現高保真紋理建模並更好地限制估計的多個視圖。對標準基準和野外圖像進行的全面實驗表明，HumanSplat在實現逼真新視角合成方面超越了現有的最先進方法。

English

Despite recent advancements in high-fidelity human reconstruction techniques, the requirements for densely captured images or time-consuming per-instance optimization significantly hinder their applications in broader scenarios. To tackle these issues, we present HumanSplat which predicts the 3D Gaussian Splatting properties of any human from a single input image in a generalizable manner. In particular, HumanSplat comprises a 2D multi-view diffusion model and a latent reconstruction transformer with human structure priors that adeptly integrate geometric priors and semantic features within a unified framework. A hierarchical loss that incorporates human semantic information is further designed to achieve high-fidelity texture modeling and better constrain the estimated multiple views. Comprehensive experiments on standard benchmarks and in-the-wild images demonstrate that HumanSplat surpasses existing state-of-the-art methods in achieving photorealistic novel-view synthesis.