GANeRF：利用鉴别器优化神经辐射场

摘要

神经辐射场（NeRF）展示了令人印象深刻的新视角合成结果；然而，即使是详尽的记录也会在重建过程中出现缺陷，例如由于观察不足的区域或轻微的光照变化。我们的目标是通过联合解决方案减轻来自各种来源的这些缺陷：我们利用生成对抗网络（GANs）生成逼真图像的能力，并将其用于增强NeRF在3D场景重建中的逼真度。为此，我们利用对抗鉴别器学习场景的补丁分布，为辐射场重建提供反馈，从而以3D一致的方式提高逼真度。因此，通过施加多视角路径渲染约束，直接修复底层3D表示中的渲染伪影。此外，我们使用多分辨率NeRF渲染来调节生成器，经过对抗训练以进一步提高渲染质量。我们展示了我们的方法显著提高了渲染质量，例如，在Tanks和Temples这些先进的室内场景中，与Nerfacto相比，LPIPS分数几乎减半，同时将PSNR提高了1.4dB。

English

Neural Radiance Fields (NeRF) have shown impressive novel view synthesis results; nonetheless, even thorough recordings yield imperfections in reconstructions, for instance due to poorly observed areas or minor lighting changes. Our goal is to mitigate these imperfections from various sources with a joint solution: we take advantage of the ability of generative adversarial networks (GANs) to produce realistic images and use them to enhance realism in 3D scene reconstruction with NeRFs. To this end, we learn the patch distribution of a scene using an adversarial discriminator, which provides feedback to the radiance field reconstruction, thus improving realism in a 3D-consistent fashion. Thereby, rendering artifacts are repaired directly in the underlying 3D representation by imposing multi-view path rendering constraints. In addition, we condition a generator with multi-resolution NeRF renderings which is adversarially trained to further improve rendering quality. We demonstrate that our approach significantly improves rendering quality, e.g., nearly halving LPIPS scores compared to Nerfacto while at the same time improving PSNR by 1.4dB on the advanced indoor scenes of Tanks and Temples.

GANeRF：利用鉴别器优化神经辐射场

GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields

摘要

Support