轻平面：神经三维场的高度可扩展组件
Lightplane: Highly-Scalable Components for Neural 3D Fields
April 30, 2024
作者: Ang Cao, Justin Johnson, Andrea Vedaldi, David Novotny
cs.AI
摘要
当代3D研究，尤其是在重建和生成方面，严重依赖于2D图像作为输入或监督。然而，目前针对这些2D-3D映射的设计具有内存密集型，给现有方法带来了重大瓶颈，并阻碍了新的应用。为此，我们提出了一对高度可扩展的组件用于3D神经场：Lightplane Render和Splatter，显著减少了2D-3D映射中的内存使用。这些创新使得能够以较小的内存和计算成本处理更多和更高分辨率的图像。我们展示了它们在各种应用中的实用性，从受益于具有图像级损失的单场景优化到实现用于大幅扩展3D重建和生成的多功能流水线。源代码：
https://github.com/facebookresearch/lightplane.
English
Contemporary 3D research, particularly in reconstruction and generation,
heavily relies on 2D images for inputs or supervision. However, current designs
for these 2D-3D mapping are memory-intensive, posing a significant bottleneck
for existing methods and hindering new applications. In response, we propose a
pair of highly scalable components for 3D neural fields: Lightplane Render and
Splatter, which significantly reduce memory usage in 2D-3D mapping. These
innovations enable the processing of vastly more and higher resolution images
with small memory and computational costs. We demonstrate their utility in
various applications, from benefiting single-scene optimization with
image-level losses to realizing a versatile pipeline for dramatically scaling
3D reconstruction and generation. Code:
https://github.com/facebookresearch/lightplane.