用于高效神经辐射场渲染的自适应外壳

摘要

神经辐射场在新视角合成方面取得了空前的质量，但其体积形式仍然昂贵，需要大量样本来渲染高分辨率图像。体积编码对于表示模糊几何体如植被和头发至关重要，并且非常适合随机优化。然而，许多场景最终主要由固体表面组成，可以通过每像素单个样本准确渲染。基于这一观点，我们提出了一种神经辐射场公式，可以在体积和基于表面的渲染之间平滑过渡，极大加速渲染速度，甚至提高视觉保真度。我们的方法构建了一个明确的网格包络，空间限定了神经体积表示。在固体区域，包络几乎收敛到一个表面，并且通常可以用单个样本渲染。为此，我们使用一个学习的空间变化核大小来推广NeuS公式，该核大小编码了密度的扩散，对体积状区域拟合宽核，对表面状区域拟合紧核。然后，我们提取一个狭窄带围绕表面的明确网格，带宽由核大小确定，并在此带内微调辐射场。在推断时，我们对网格投射光线，并仅在封闭区域内评估辐射场，大大减少了所需的样本数量。实验表明，我们的方法实现了高保真度的高效渲染。我们还展示了提取的包络使得诸如动画和模拟等下游应用成为可能。

English

Neural radiance fields achieve unprecedented quality for novel view synthesis, but their volumetric formulation remains expensive, requiring a huge number of samples to render high-resolution images. Volumetric encodings are essential to represent fuzzy geometry such as foliage and hair, and they are well-suited for stochastic optimization. Yet, many scenes ultimately consist largely of solid surfaces which can be accurately rendered by a single sample per pixel. Based on this insight, we propose a neural radiance formulation that smoothly transitions between volumetric- and surface-based rendering, greatly accelerating rendering speed and even improving visual fidelity. Our method constructs an explicit mesh envelope which spatially bounds a neural volumetric representation. In solid regions, the envelope nearly converges to a surface and can often be rendered with a single sample. To this end, we generalize the NeuS formulation with a learned spatially-varying kernel size which encodes the spread of the density, fitting a wide kernel to volume-like regions and a tight kernel to surface-like regions. We then extract an explicit mesh of a narrow band around the surface, with width determined by the kernel size, and fine-tune the radiance field within this band. At inference time, we cast rays against the mesh and evaluate the radiance field only within the enclosed region, greatly reducing the number of samples required. Experiments show that our approach enables efficient rendering at very high fidelity. We also demonstrate that the extracted envelope enables downstream applications such as animation and simulation.

用于高效神经辐射场渲染的自适应外壳

Adaptive Shells for Efficient Neural Radiance Field Rendering

摘要

Support