用於高效3D場景表示的重建潛在空間神經輻射場

摘要

神經輻射場（Neural Radiance Fields，NeRFs）已被證明是強大的3D表示形式，能夠高質量地合成複雜場景的新視角。雖然NeRFs已應用於圖形、視覺和機器人領域，但慢速渲染和特徵視覺異常問題阻礙了在許多應用案例中的應用。在本研究中，我們探討將自編碼器（Autoencoder，AE）與NeRF結合，其中渲染潛在特徵（而非顏色），然後進行卷積解碼。由此產生的潛在空間NeRF可以比標準顏色空間NeRF產生更高質量的新視角，因為自編碼器可以糾正某些視覺異常，同時渲染速度提高了三倍以上。我們的工作與其他改善NeRF效率的技術是正交的。此外，通過縮小自編碼器架構，我們可以控制效率和圖像質量之間的折衷，實現渲染速度提高13倍以上，僅有輕微性能下降。我們希望我們的方法可以成為下游任務的高效且高保真3D場景表示的基礎，尤其是在需要保留可微性的許多機器人場景中。

English

Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations, capable of high quality novel view synthesis of complex scenes. While NeRFs have been applied to graphics, vision, and robotics, problems with slow rendering speed and characteristic visual artifacts prevent adoption in many use cases. In this work, we investigate combining an autoencoder (AE) with a NeRF, in which latent features (instead of colours) are rendered and then convolutionally decoded. The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs, as the AE can correct certain visual artifacts, while rendering over three times faster. Our work is orthogonal to other techniques for improving NeRF efficiency. Further, we can control the tradeoff between efficiency and image quality by shrinking the AE architecture, achieving over 13 times faster rendering with only a small drop in performance. We hope that our approach can form the basis of an efficient, yet high-fidelity, 3D scene representation for downstream tasks, especially when retaining differentiability is useful, as in many robotics scenarios requiring continual learning.

用於高效3D場景表示的重建潛在空間神經輻射場

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

摘要

Support