用于高效三维场景表示的重建潜空间神经辐射场

摘要

神经辐射场（NeRFs）已被证明是强大的3D表示形式，能够高质量地合成复杂场景的新视角。虽然NeRFs已应用于图形、视觉和机器人领域，但缓慢的渲染速度和特征性视觉伪影问题阻碍了其在许多应用场景中的采用。在这项工作中，我们研究将自动编码器（AE）与NeRF相结合，其中渲染潜在特征（而非颜色），然后进行卷积解码。由此产生的潜在空间NeRF可以比标准颜色空间NeRF产生更高质量的新视角，因为AE可以纠正某些视觉伪影，同时渲染速度提高了三倍以上。我们的工作与其他改进NeRF效率的技术是正交的。此外，通过缩小AE架构，我们可以控制效率和图像质量之间的权衡，仅在性能略微下降的情况下实现超过13倍的更快渲染速度。我们希望我们的方法能够成为下游任务的高效而高保真的3D场景表示的基础，特别是在需要保持可区分性的许多需要持续学习的机器人场景中。

English

Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations, capable of high quality novel view synthesis of complex scenes. While NeRFs have been applied to graphics, vision, and robotics, problems with slow rendering speed and characteristic visual artifacts prevent adoption in many use cases. In this work, we investigate combining an autoencoder (AE) with a NeRF, in which latent features (instead of colours) are rendered and then convolutionally decoded. The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs, as the AE can correct certain visual artifacts, while rendering over three times faster. Our work is orthogonal to other techniques for improving NeRF efficiency. Further, we can control the tradeoff between efficiency and image quality by shrinking the AE architecture, achieving over 13 times faster rendering with only a small drop in performance. We hope that our approach can form the basis of an efficient, yet high-fidelity, 3D scene representation for downstream tasks, especially when retaining differentiability is useful, as in many robotics scenarios requiring continual learning.

用于高效三维场景表示的重建潜空间神经辐射场

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

摘要

Support