实时神经外观模型

摘要

我们提出了一个完整的系统，用于实时渲染先前仅用于离线使用的具有复杂外观的场景。这是通过算法和系统级创新的结合实现的。我们的外观模型利用了学习的分层纹理，通过神经解码器进行解释，产生反射值和重要采样方向。为了最大限度地利用解码器的建模能力，我们为解码器配备了两个图形先验。第一个先验——将方向转换为学习的着色框架——有助于准确重建中尺度效应。第二个先验——微平面采样分布——允许神经解码器高效执行重要采样。由此产生的外观模型支持各向异性采样和层次细节渲染，并允许将深度分层材质图层烘焙成紧凑的统一神经表示。通过将硬件加速的张量操作暴露给光线追踪着色器，我们展示了在实时路径追踪器内高效内联和执行神经解码器的可能性。我们分析了随着神经材质数量增加而增加的可扩展性，并提出使用针对一致和分歧执行优化的代码来提高性能。我们的神经材质着色器可以比非神经分层材质快一个数量级。这为在游戏和实时预览等实时应用中使用电影质量的视觉效果打开了大门。

English

We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior -- transformation of directions into learned shading frames -- facilitates accurate reconstruction of mesoscale effects. The second prior -- a microfacet sampling distribution -- allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.

实时神经外观模型

Real-Time Neural Appearance Models

摘要

Support