Snap-Snap：双图速成，毫秒级重建3D人体高斯模型

摘要

从稀疏视角重建三维人体一直是一个引人注目的研究课题，这对于拓宽相关应用领域至关重要。本文提出了一项极具挑战性但价值显著的任务：仅通过正反两张图像重建人体模型，这大大降低了用户创建自身三维数字人的门槛。该任务的主要挑战在于如何构建三维一致性并从高度稀疏的输入中恢复缺失信息。我们基于基础重建模型重新设计了几何重建模型，即便输入图像重叠区域极少，也能通过大量人体数据训练预测出一致的点云。此外，采用增强算法补充缺失的色彩信息，从而获得完整的带色彩人体点云，这些点云可直接转化为三维高斯分布以提升渲染质量。实验表明，在单张NVIDIA RTX 4090显卡上，我们的方法能以190毫秒的速度重建完整人体，处理两张分辨率为1024x1024的图像，在THuman2.0及跨域数据集上展现了最先进的性能。更重要的是，即便使用低成本移动设备拍摄的图像，我们的方法也能完成人体重建，降低了对数据采集的要求。演示视频及代码已发布于https://hustvl.github.io/Snap-Snap/。

English

Reconstructing 3D human bodies from sparse views has been an appealing topic, which is crucial to broader the related applications. In this paper, we propose a quite challenging but valuable task to reconstruct the human body from only two images, i.e., the front and back view, which can largely lower the barrier for users to create their own 3D digital humans. The main challenges lie in the difficulty of building 3D consistency and recovering missing information from the highly sparse input. We redesign a geometry reconstruction model based on foundation reconstruction models to predict consistent point clouds even input images have scarce overlaps with extensive human data training. Furthermore, an enhancement algorithm is applied to supplement the missing color information, and then the complete human point clouds with colors can be obtained, which are directly transformed into 3D Gaussians for better rendering quality. Experiments show that our method can reconstruct the entire human in 190 ms on a single NVIDIA RTX 4090, with two images at a resolution of 1024x1024, demonstrating state-of-the-art performance on the THuman2.0 and cross-domain datasets. Additionally, our method can complete human reconstruction even with images captured by low-cost mobile devices, reducing the requirements for data collection. Demos and code are available at https://hustvl.github.io/Snap-Snap/.

Snap-Snap：双图速成，毫秒级重建3D人体高斯模型

Snap-Snap: Taking Two Images to Reconstruct 3D Human Gaussians in Milliseconds

摘要

Support