PlatoNeRF：透過單視角雙反射激光雷達在柏拉圖洞穴中進行的3D重建

摘要

從單視圖進行的三維重建具有挑戰性，因為存在單眼線索的模棱兩可性以及有關遮蔽區域的信息不足。神經輻射場（NeRF）雖然在視角合成和三維重建方面很受歡迎，但通常依賴多視圖影像。現有的使用NeRF進行單視圖三維重建的方法要麼依賴於數據先驗來虛構遮蔽區域的視角，但這可能不具有物理準確性，要麼依賴於RGB相機觀察到的陰影，但在環境光線和低反照率背景下很難檢測。我們提出使用由單光子雪崩二極管捕獲的飛行時間數據來克服這些限制。我們的方法使用激光雷達瞬態數據監督NeRF模擬兩次反射的光路。通過利用NeRF和激光雷達測量的兩次反射光的優勢，我們展示了可以在沒有數據先驗或依賴受控環境照明或場景反照率的情況下重建可見和遮蔽幾何。此外，我們展示了在傳感器空間和時間分辨率的實際限制下改進的泛化能力。我們認為，隨著單光子激光雷達在消費者設備（如手機、平板電腦和頭戴設備）上變得普及，我們的方法是一個有前途的方向。

English

3D reconstruction from a single-view is challenging because of the ambiguity from monocular cues and lack of information about occluded regions. Neural radiance fields (NeRF), while popular for view synthesis and 3D reconstruction, are typically reliant on multi-view images. Existing methods for single-view 3D reconstruction with NeRF rely on either data priors to hallucinate views of occluded regions, which may not be physically accurate, or shadows observed by RGB cameras, which are difficult to detect in ambient light and low albedo backgrounds. We propose using time-of-flight data captured by a single-photon avalanche diode to overcome these limitations. Our method models two-bounce optical paths with NeRF, using lidar transient data for supervision. By leveraging the advantages of both NeRF and two-bounce light measured by lidar, we demonstrate that we can reconstruct visible and occluded geometry without data priors or reliance on controlled ambient lighting or scene albedo. In addition, we demonstrate improved generalization under practical constraints on sensor spatial- and temporal-resolution. We believe our method is a promising direction as single-photon lidars become ubiquitous on consumer devices, such as phones, tablets, and headsets.

PlatoNeRF：透過單視角雙反射激光雷達在柏拉圖洞穴中進行的3D重建

PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar

摘要

Support