通过学习辐射传输梯度进行神经元照明和次表面散射。

Neural Relighting with Subsurface Scattering by Learning the Radiance Transfer Gradient

June 15, 2023

作者: Shizhan Zhu, Shunsuke Saito, Aljaz Bozic, Carlos Aliaga, Trevor Darrell, Christop Lassner

cs.AI

摘要

在不同光照条件下重建和重新照明物体和场景具有挑战性：现有的神经渲染方法通常无法处理材料和光线之间复杂的相互作用。将预先计算的辐射传输技术纳入其中可以实现全局光照，但仍然难以处理具有次表面散射效应的材料。我们提出了一个新颖的框架，通过体积渲染学习辐射传输场，并利用各种外观线索来端到端地优化几何。该框架扩展了照明和重建能力，以以数据驱动的方式处理更广泛范围的材料。由此产生的模型在现有和新颖条件下产生合理的渲染结果。我们将公开发布我们的代码和一个包含具有次表面散射效应的物体的新颖光照舞台数据集。

English

Reconstructing and relighting objects and scenes under varying lighting conditions is challenging: existing neural rendering methods often cannot handle the complex interactions between materials and light. Incorporating pre-computed radiance transfer techniques enables global illumination, but still struggles with materials with subsurface scattering effects. We propose a novel framework for learning the radiance transfer field via volume rendering and utilizing various appearance cues to refine geometry end-to-end. This framework extends relighting and reconstruction capabilities to handle a wider range of materials in a data-driven fashion. The resulting models produce plausible rendering results in existing and novel conditions. We will release our code and a novel light stage dataset of objects with subsurface scattering effects publicly available.