神經方向編碼用於高效且準確的視角相依外觀建模

Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling

May 23, 2024

作者: Liwen Wu, Sai Bi, Zexiang Xu, Fujun Luan, Kai Zhang, Iliyan Georgiev, Kalyan Sunkavalli, Ravi Ramamoorthi

cs.AI

摘要

對於像是閃亮金屬或光澤油漆等具有鏡面特性的物體進行新視角合成仍然是一個重大挑戰。不僅是物體的光澤外觀，還有包括環境中其他物體的反射在內的全局照明效應，都是忠實再現場景所必需的關鍵元素。在本文中，我們提出了神經方向編碼（NDE），這是一種基於視角的外觀編碼方法，用於渲染具有鏡面特性的物體的神經輻射場（NeRF）。NDE將基於特徵網格的空間編碼概念轉移到角度領域，顯著提高了對高頻角度信號進行建模的能力。與先前僅使用角度輸入的編碼函數不同，我們額外地錐追蹤空間特徵，以獲得一種空間變化的方向編碼，這解決了具有挑戰性的互反射效應。對合成和真實數據集進行的大量實驗表明，具有NDE的NeRF模型（1）在鏡面物體的視角合成方面優於當前技術水平，並且（2）可使用小型網絡實現快速（實時）推斷。項目網頁和源代碼可在以下網址找到：https://lwwu2.github.io/nde/。

English

Novel-view synthesis of specular objects like shiny metals or glossy paints remains a significant challenge. Not only the glossy appearance but also global illumination effects, including reflections of other objects in the environment, are critical components to faithfully reproduce a scene. In this paper, we present Neural Directional Encoding (NDE), a view-dependent appearance encoding of neural radiance fields (NeRF) for rendering specular objects. NDE transfers the concept of feature-grid-based spatial encoding to the angular domain, significantly improving the ability to model high-frequency angular signals. In contrast to previous methods that use encoding functions with only angular input, we additionally cone-trace spatial features to obtain a spatially varying directional encoding, which addresses the challenging interreflection effects. Extensive experiments on both synthetic and real datasets show that a NeRF model with NDE (1) outperforms the state of the art on view synthesis of specular objects, and (2) works with small networks to allow fast (real-time) inference. The project webpage and source code are available at: https://lwwu2.github.io/nde/.