鏡像-NeRF：使用Whitted風格的射線追踪學習鏡像的神經輻射場

摘要

最近，神經輝度場（Neural Radiance Fields，NeRF）在新視角合成、表面重建等方面取得了顯著成功。然而，由於其渲染流程中未考慮物理反射，NeRF將鏡子中的反射誤認為獨立的虛擬場景，導致鏡子的重建不準確，並在鏡子中出現多視角不一致的反射。本文提出了一種新穎的神經渲染框架，名為Mirror-NeRF，能夠學習鏡子的準確幾何和反射，並支持各種與鏡子相關的場景操作應用，如將新物體或鏡子添加到場景中，並在鏡子中合成這些新物體的反射，控制鏡子的粗糙度等。為實現此目標，我們提出了一個統一的輝度場，引入了反射概率並追踪光線，遵循Whitted Ray Tracing的光傳輸模型，並開發了幾種技術來促進學習過程。在合成和真實數據集上進行的實驗和比較顯示了我們方法的優越性。代碼和補充材料可在項目網頁上找到：https://zju3dv.github.io/Mirror-NeRF/.

English

Recently, Neural Radiance Fields (NeRF) has exhibited significant success in novel view synthesis, surface reconstruction, etc. However, since no physical reflection is considered in its rendering pipeline, NeRF mistakes the reflection in the mirror as a separate virtual scene, leading to the inaccurate reconstruction of the mirror and multi-view inconsistent reflections in the mirror. In this paper, we present a novel neural rendering framework, named Mirror-NeRF, which is able to learn accurate geometry and reflection of the mirror and support various scene manipulation applications with mirrors, such as adding new objects or mirrors into the scene and synthesizing the reflections of these new objects in mirrors, controlling mirror roughness, etc. To achieve this goal, we propose a unified radiance field by introducing the reflection probability and tracing rays following the light transport model of Whitted Ray Tracing, and also develop several techniques to facilitate the learning process. Experiments and comparisons on both synthetic and real datasets demonstrate the superiority of our method. The code and supplementary material are available on the project webpage: https://zju3dv.github.io/Mirror-NeRF/.

鏡像-NeRF：使用Whitted風格的射線追踪學習鏡像的神經輻射場

Mirror-NeRF: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing

摘要

Support