GANeRF：利用鑑別器優化神經輻射場

摘要

神經輻射場（Neural Radiance Fields，NeRF）展示了令人印象深刻的新視角合成結果；然而，即使是詳盡的記錄也會在重建中產生瑕疵，例如由於觀察不足的區域或輕微的光線變化。我們的目標是通過聯合解決方案來減輕來自各種來源的這些瑕疵：我們利用生成對抗網絡（Generative Adversarial Networks，GANs）生成逼真圖像的能力，並將其用於通過NeRF增強3D場景重建的逼真度。為此，我們利用對抗鑑別器學習場景的補丁分佈，該鑑別器提供反饋以改進輻射場重建，從而以3D一致的方式提高逼真度。因此，通過施加多視角路徑渲染約束，渲染藝術品直接在基礎3D表示中得到修復。此外，我們條件一個生成器使用多分辨率NeRF渲染，通過對抗訓練進一步提高渲染質量。我們展示了我們的方法顯著提高了渲染質量，例如，與Nerfacto相比，LPIPS分數幾乎減半，同時在Tanks and Temples進階室內場景上將PSNR提高了1.4dB。

English

Neural Radiance Fields (NeRF) have shown impressive novel view synthesis results; nonetheless, even thorough recordings yield imperfections in reconstructions, for instance due to poorly observed areas or minor lighting changes. Our goal is to mitigate these imperfections from various sources with a joint solution: we take advantage of the ability of generative adversarial networks (GANs) to produce realistic images and use them to enhance realism in 3D scene reconstruction with NeRFs. To this end, we learn the patch distribution of a scene using an adversarial discriminator, which provides feedback to the radiance field reconstruction, thus improving realism in a 3D-consistent fashion. Thereby, rendering artifacts are repaired directly in the underlying 3D representation by imposing multi-view path rendering constraints. In addition, we condition a generator with multi-resolution NeRF renderings which is adversarially trained to further improve rendering quality. We demonstrate that our approach significantly improves rendering quality, e.g., nearly halving LPIPS scores compared to Nerfacto while at the same time improving PSNR by 1.4dB on the advanced indoor scenes of Tanks and Temples.

GANeRF：利用鑑別器優化神經輻射場

GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields

摘要

Support