镜像-NeRF：使用Whitted风格的光线追踪学习镜子的神经辐射场

摘要

最近，神经辐射场（Neural Radiance Fields，NeRF）在新视角合成、表面重建等方面取得了显著成功。然而，由于其渲染流程中未考虑物理反射，NeRF将镜子中的反射错误地视为单独的虚拟场景，导致镜子的重建不准确以及镜子中多视角反射不一致。本文提出了一种新颖的神经渲染框架，名为Mirror-NeRF，能够学习镜子的准确几何和反射，并支持各种镜子场景操作应用，如在场景中添加新对象或镜子，合成这些新对象在镜子中的反射，控制镜子的粗糙度等。为实现这一目标，我们提出了一个统一的辐射场，引入了反射概率，并沿着Whitted光线追踪模型追踪光线，同时开发了几种技术来促进学习过程。在合成和真实数据集上的实验证明了我们方法的优越性。代码和补充材料可在项目网页上找到：https://zju3dv.github.io/Mirror-NeRF/。

English

Recently, Neural Radiance Fields (NeRF) has exhibited significant success in novel view synthesis, surface reconstruction, etc. However, since no physical reflection is considered in its rendering pipeline, NeRF mistakes the reflection in the mirror as a separate virtual scene, leading to the inaccurate reconstruction of the mirror and multi-view inconsistent reflections in the mirror. In this paper, we present a novel neural rendering framework, named Mirror-NeRF, which is able to learn accurate geometry and reflection of the mirror and support various scene manipulation applications with mirrors, such as adding new objects or mirrors into the scene and synthesizing the reflections of these new objects in mirrors, controlling mirror roughness, etc. To achieve this goal, we propose a unified radiance field by introducing the reflection probability and tracing rays following the light transport model of Whitted Ray Tracing, and also develop several techniques to facilitate the learning process. Experiments and comparisons on both synthetic and real datasets demonstrate the superiority of our method. The code and supplementary material are available on the project webpage: https://zju3dv.github.io/Mirror-NeRF/.

镜像-NeRF：使用Whitted风格的光线追踪学习镜子的神经辐射场

Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing

摘要

Support