神经方向编码用于高效和准确的视角相关外观建模。

Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling

May 23, 2024

作者: Liwen Wu, Sai Bi, Zexiang Xu, Fujun Luan, Kai Zhang, Iliyan Georgiev, Kalyan Sunkavalli, Ravi Ramamoorthi

cs.AI

摘要

对于诸如闪亮金属或光泽油漆等具有镜面特性的物体进行新视角合成仍然是一个重大挑战。不仅需要考虑光泽外观，还需要全局照明效果，包括环境中其他物体的反射，这些都是忠实再现场景所必需的关键组成部分。在本文中，我们提出了神经方向编码（NDE），这是一种基于视角的外观编码，用于呈现具有镜面特性的物体的神经辐射场（NeRF）。NDE将基于特征网格的空间编码概念转移到角度域，显著提高了对高频角信号建模的能力。与先前仅使用角度输入的编码函数不同，我们还锥追踪空间特征，以获得空间变化的方向编码，从而解决了具有挑战性的互反射效应。对合成和真实数据集的大量实验表明，具有NDE的NeRF模型（1）在镜面物体视角合成方面优于现有技术水平，（2）能够通过小型网络实现快速（实时）推断。项目网页和源代码可在以下网址找到：https://lwwu2.github.io/nde/。

English

Novel-view synthesis of specular objects like shiny metals or glossy paints remains a significant challenge. Not only the glossy appearance but also global illumination effects, including reflections of other objects in the environment, are critical components to faithfully reproduce a scene. In this paper, we present Neural Directional Encoding (NDE), a view-dependent appearance encoding of neural radiance fields (NeRF) for rendering specular objects. NDE transfers the concept of feature-grid-based spatial encoding to the angular domain, significantly improving the ability to model high-frequency angular signals. In contrast to previous methods that use encoding functions with only angular input, we additionally cone-trace spatial features to obtain a spatially varying directional encoding, which addresses the challenging interreflection effects. Extensive experiments on both synthetic and real datasets show that a NeRF model with NDE (1) outperforms the state of the art on view synthesis of specular objects, and (2) works with small networks to allow fast (real-time) inference. The project webpage and source code are available at: https://lwwu2.github.io/nde/.