PlatoNeRF：通过单视角双弹跳激光雷达在柏拉图洞穴中进行的3D重建

摘要

由单视角进行的三维重建具有挑战性，因为存在单眼线索的模糊性以及关于遮挡区域缺乏信息。神经辐射场（NeRF）虽然在视图合成和三维重建中很受欢迎，但通常依赖于多视图图像。现有的利用NeRF进行单视角三维重建的方法要么依赖于数据先验来虚拟遮挡区域的视图，这可能不够物理准确，要么依赖于RGB相机观察到的阴影，但在环境光线和低反照率背景下很难检测到。我们提出使用由单光子雪崩二极管捕获的飞行时间数据来克服这些限制。我们的方法使用激光雷达瞬态数据监督，用NeRF模拟两次光学路径。通过利用NeRF和激光雷达测量的两次光路的优势，我们展示了可以重建可见和遮挡几何形状，而无需数据先验或依赖于受控环境照明或场景反照率。此外，我们展示了在传感器空间和时间分辨率受到实际约束时的改进泛化能力。我们相信随着单光子激光雷达在消费设备（如手机、平板电脑和头戴设备）上变得普遍，我们的方法是一个有前途的方向。

English

3D reconstruction from a single-view is challenging because of the ambiguity from monocular cues and lack of information about occluded regions. Neural radiance fields (NeRF), while popular for view synthesis and 3D reconstruction, are typically reliant on multi-view images. Existing methods for single-view 3D reconstruction with NeRF rely on either data priors to hallucinate views of occluded regions, which may not be physically accurate, or shadows observed by RGB cameras, which are difficult to detect in ambient light and low albedo backgrounds. We propose using time-of-flight data captured by a single-photon avalanche diode to overcome these limitations. Our method models two-bounce optical paths with NeRF, using lidar transient data for supervision. By leveraging the advantages of both NeRF and two-bounce light measured by lidar, we demonstrate that we can reconstruct visible and occluded geometry without data priors or reliance on controlled ambient lighting or scene albedo. In addition, we demonstrate improved generalization under practical constraints on sensor spatial- and temporal-resolution. We believe our method is a promising direction as single-photon lidars become ubiquitous on consumer devices, such as phones, tablets, and headsets.

PlatoNeRF：通过单视角双弹跳激光雷达在柏拉图洞穴中进行的3D重建

PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar

摘要

Support