利用空間雕刻外繪技術從單張影像中重建360度圖像

摘要

我們介紹了一個新穎的框架 POP3D，可以從單張圖像創建完整的 360° 全景 3D 模型。POP3D 解決了限制單視角重建的兩個突出問題。首先，POP3D 提供了對任意類別的顯著泛化能力，這是先前方法難以實現的特點。其次，POP3D 進一步提高了重建的保真度和自然性，這是同期工作所欠缺的關鍵方面。我們的方法結合了四個主要組件的優勢：(1) 單眼深度和法向預測器，用於預測關鍵的幾何線索，(2) 一種能夠劃分目標對象可能看不見部分的空間雕刻方法，(3) 在大規模圖像數據集上預先訓練的生成模型，可以完成目標的看不見區域，以及 (4) 一種神經隱式表面重建方法，專門用於使用 RGB 圖像和單眼幾何線索重建物體。這些組件的組合使得 POP3D 能夠輕鬆泛化到各種野外圖像並生成最先進的重建結果，明顯優於類似工作。項目頁面：http://cg.postech.ac.kr/research/POP3D

English

We introduce POP3D, a novel framework that creates a full 360^circ-view 3D model from a single image. POP3D resolves two prominent issues that limit the single-view reconstruction. Firstly, POP3D offers substantial generalizability to arbitrary categories, a trait that previous methods struggle to achieve. Secondly, POP3D further improves reconstruction fidelity and naturalness, a crucial aspect that concurrent works fall short of. Our approach marries the strengths of four primary components: (1) a monocular depth and normal predictor that serves to predict crucial geometric cues, (2) a space carving method capable of demarcating the potentially unseen portions of the target object, (3) a generative model pre-trained on a large-scale image dataset that can complete unseen regions of the target, and (4) a neural implicit surface reconstruction method tailored in reconstructing objects using RGB images along with monocular geometric cues. The combination of these components enables POP3D to readily generalize across various in-the-wild images and generate state-of-the-art reconstructions, outperforming similar works by a significant margin. Project page: http://cg.postech.ac.kr/research/POP3D

利用空間雕刻外繪技術從單張影像中重建360度圖像

360^circ Reconstruction From a Single Image Using Space Carved Outpainting

摘要

Support