MixRT：混合神經表示用於實時 NeRF 渲染

摘要

神經輻射場（Neural Radiance Field，NeRF）已成為新視角合成的領先技術，憑藉其令人印象深刻的照片逼真重建和渲染能力。然而，在大型場景中實現即時的 NeRF 渲染一直存在挑戰，通常需要採用要麼是包含大量三角形的複雜烘焙網格表示，要麼是資源密集的烘焙表示中的射線逐步。我們挑戰這些慣例，觀察到高質量幾何，用包含大量三角形的網格表示，對於實現照片逼真的渲染質量並非必要。因此，我們提出了 MixRT，一種新型的 NeRF 表示，包括低質量網格、視角相依位移圖和壓縮的 NeRF 模型。這種設計有效地利用現有圖形硬體的能力，從而實現了邊緣設備上的即時 NeRF 渲染。通過高度優化的基於 WebGL 的渲染框架，我們提出的 MixRT 在邊緣設備上實現了即時渲染速度（在 MacBook M1 Pro 筆記本電腦上以 1280 x 720 的分辨率超過 30 FPS），更好的渲染質量（在 Unbounded-360 數據集的室內場景中高出 0.2 PSNR），以及更小的存儲大小（與最先進的方法相比少於 80%）。

English

Neural Radiance Field (NeRF) has emerged as a leading technique for novel view synthesis, owing to its impressive photorealistic reconstruction and rendering capability. Nevertheless, achieving real-time NeRF rendering in large-scale scenes has presented challenges, often leading to the adoption of either intricate baked mesh representations with a substantial number of triangles or resource-intensive ray marching in baked representations. We challenge these conventions, observing that high-quality geometry, represented by meshes with substantial triangles, is not necessary for achieving photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF representation that includes a low-quality mesh, a view-dependent displacement map, and a compressed NeRF model. This design effectively harnesses the capabilities of existing graphics hardware, thus enabling real-time NeRF rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering framework, our proposed MixRT attains real-time rendering speeds on edge devices (over 30 FPS at a resolution of 1280 x 720 on a MacBook M1 Pro laptop), better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360 datasets), and a smaller storage size (less than 80% compared to state-of-the-art methods).

MixRT：混合神經表示用於實時 NeRF 渲染

MixRT: Mixed Neural Representations For Real-Time NeRF Rendering

摘要

Support