MixRT：混合神经表示用于实时NeRF渲染。

摘要

神经辐射场（NeRF）已成为新视角合成的领先技术，因其令人印象深刻的逼真重建和渲染能力而闻名。然而，在大规模场景中实现实时NeRF渲染存在挑战，通常需要采用要么复杂的烘焙网格表示，要么资源密集的烘焙表示中的光线行进，我们挑战这些传统，观察到高质量几何，用大量三角形表示的网格并非实现逼真渲染质量所必需。因此，我们提出了MixRT，一种新颖的NeRF表示，包括低质量网格、视角相关位移图和压缩的NeRF模型。这种设计有效地利用了现有图形硬件的能力，从而实现了边缘设备上的实时NeRF渲染。利用高度优化的基于WebGL的渲染框架，我们提出的MixRT在边缘设备上实现了实时渲染速度（在MacBook M1 Pro笔记本上以1280 x 720分辨率超过30 FPS），更好的渲染质量（在Unbounded-360数据集的室内场景中高出0.2 PSNR），以及更小的存储空间（与最先进方法相比少于80%）。

English

Neural Radiance Field (NeRF) has emerged as a leading technique for novel view synthesis, owing to its impressive photorealistic reconstruction and rendering capability. Nevertheless, achieving real-time NeRF rendering in large-scale scenes has presented challenges, often leading to the adoption of either intricate baked mesh representations with a substantial number of triangles or resource-intensive ray marching in baked representations. We challenge these conventions, observing that high-quality geometry, represented by meshes with substantial triangles, is not necessary for achieving photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF representation that includes a low-quality mesh, a view-dependent displacement map, and a compressed NeRF model. This design effectively harnesses the capabilities of existing graphics hardware, thus enabling real-time NeRF rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering framework, our proposed MixRT attains real-time rendering speeds on edge devices (over 30 FPS at a resolution of 1280 x 720 on a MacBook M1 Pro laptop), better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360 datasets), and a smaller storage size (less than 80% compared to state-of-the-art methods).

MixRT：混合神经表示用于实时NeRF渲染。

MixRT: Mixed Neural Representations For Real-Time NeRF Rendering

摘要

Support