Snap-Snap：透過兩張影像在毫秒內重建3D人體高斯模型

摘要

從稀疏視角重建3D人體一直是一個引人入勝的研究主題，這對於拓展相關應用至關重要。本文提出了一項極具挑戰性但價值重大的任務：僅憑兩張圖像（即正面和背面視圖）重建人體，這將大幅降低用戶創建自身3D數字人體的門檻。主要挑戰在於如何建立3D一致性以及從極度稀疏的輸入中恢復缺失信息。我們基於基礎重建模型重新設計了一種幾何重建模型，即使輸入圖像之間重疊區域極少，也能通過大量人體數據訓練預測出一致性的點雲。此外，應用了一種增強算法來補充缺失的色彩信息，從而獲得帶有色彩的完整人體點雲，這些點雲可直接轉化為3D高斯分佈以提升渲染質量。實驗表明，我們的方法在單張NVIDIA RTX 4090上僅需190毫秒即可完成整個人體重建，處理兩張分辨率為1024x1024的圖像，在THuman2.0及跨域數據集上展現了頂尖性能。此外，即便使用低成本移動設備拍攝的圖像，我們的方法也能完成人體重建，降低了數據收集的要求。演示與代碼可訪問https://hustvl.github.io/Snap-Snap/獲取。

English

Reconstructing 3D human bodies from sparse views has been an appealing topic, which is crucial to broader the related applications. In this paper, we propose a quite challenging but valuable task to reconstruct the human body from only two images, i.e., the front and back view, which can largely lower the barrier for users to create their own 3D digital humans. The main challenges lie in the difficulty of building 3D consistency and recovering missing information from the highly sparse input. We redesign a geometry reconstruction model based on foundation reconstruction models to predict consistent point clouds even input images have scarce overlaps with extensive human data training. Furthermore, an enhancement algorithm is applied to supplement the missing color information, and then the complete human point clouds with colors can be obtained, which are directly transformed into 3D Gaussians for better rendering quality. Experiments show that our method can reconstruct the entire human in 190 ms on a single NVIDIA RTX 4090, with two images at a resolution of 1024x1024, demonstrating state-of-the-art performance on the THuman2.0 and cross-domain datasets. Additionally, our method can complete human reconstruction even with images captured by low-cost mobile devices, reducing the requirements for data collection. Demos and code are available at https://hustvl.github.io/Snap-Snap/.

Snap-Snap：透過兩張影像在毫秒內重建3D人體高斯模型

Snap-Snap: Taking Two Images to Reconstruct 3D Human Gaussians in Milliseconds

摘要

Support