使用空间雕刻外描技术从单张图像进行360°重建

摘要

我们介绍了一种新颖的框架POP3D，它可以从单个图像创建完整的360°全景3D模型。POP3D解决了限制单视图重建的两个突出问题。首先，POP3D具有对任意类别的显著泛化能力，这是先前方法难以实现的特点。其次，POP3D进一步提高了重建的保真度和自然度，这是同时期作品所欠缺的关键方面。我们的方法融合了四个主要组件的优势：（1）单目深度和法线预测器，用于预测关键的几何线索，（2）空间雕刻方法，能够划分目标对象可能看不见的部分，（3）在大规模图像数据集上预训练的生成模型，可以完成目标看不见的区域，以及（4）一种神经隐式表面重建方法，专门用于使用RGB图像和单目几何线索重建对象。这些组件的结合使得POP3D能够轻松泛化到各种野外图像，并生成最先进的重建结果，明显优于类似作品。项目页面：http://cg.postech.ac.kr/research/POP3D

English

We introduce POP3D, a novel framework that creates a full 360^circ-view 3D model from a single image. POP3D resolves two prominent issues that limit the single-view reconstruction. Firstly, POP3D offers substantial generalizability to arbitrary categories, a trait that previous methods struggle to achieve. Secondly, POP3D further improves reconstruction fidelity and naturalness, a crucial aspect that concurrent works fall short of. Our approach marries the strengths of four primary components: (1) a monocular depth and normal predictor that serves to predict crucial geometric cues, (2) a space carving method capable of demarcating the potentially unseen portions of the target object, (3) a generative model pre-trained on a large-scale image dataset that can complete unseen regions of the target, and (4) a neural implicit surface reconstruction method tailored in reconstructing objects using RGB images along with monocular geometric cues. The combination of these components enables POP3D to readily generalize across various in-the-wild images and generate state-of-the-art reconstructions, outperforming similar works by a significant margin. Project page: http://cg.postech.ac.kr/research/POP3D

使用空间雕刻外描技术从单张图像进行360°重建

360^circ Reconstruction From a Single Image Using Space Carved Outpainting

摘要

Support