輕平面：用於神經三維場的高度可擴展元件

Lightplane: Highly-Scalable Components for Neural 3D Fields

April 30, 2024

作者: Ang Cao, Justin Johnson, Andrea Vedaldi, David Novotny

cs.AI

摘要

當前的3D研究，特別是在重建和生成方面，大量依賴於2D圖像作為輸入或監督。然而，目前這些2D-3D映射的設計對記憶體需求高，對現有方法構成了重大瓶頸，並阻礙了新應用的發展。為此，我們提出了一對高度可擴展的組件，用於3D神經場：Lightplane Render和Splatter，顯著降低了2D-3D映射中的記憶體使用量。這些創新使得能夠以較小的記憶體和計算成本處理更多且更高分辨率的圖像。我們展示了它們在各種應用中的實用性，從受益於單場景優化的圖像級損失，到實現一個多功能管道，大幅擴展3D重建和生成。程式碼：https://github.com/facebookresearch/lightplane。

English

Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision. However, current designs for these 2D-3D mapping are memory-intensive, posing a significant bottleneck for existing methods and hindering new applications. In response, we propose a pair of highly scalable components for 3D neural fields: Lightplane Render and Splatter, which significantly reduce memory usage in 2D-3D mapping. These innovations enable the processing of vastly more and higher resolution images with small memory and computational costs. We demonstrate their utility in various applications, from benefiting single-scene optimization with image-level losses to realizing a versatile pipeline for dramatically scaling 3D reconstruction and generation. Code: https://github.com/facebookresearch/lightplane.