即時神經外觀模型

摘要

我們提出了一個完整的系統，用於實時渲染具有複雜外觀的場景，這些場景以前僅用於離線使用。這是通過算法和系統級創新的組合來實現的。我們的外觀模型利用了通過神經解碼器解釋的學習的分層紋理，這些紋理產生反射值和重要採樣方向。為了最大限度地利用解碼器的建模能力，我們為解碼器配備了兩個圖形先驗。第一個先驗——將方向轉換為學習的著色幀——有助於準確重建中尺度效應。第二個先驗——微平面採樣分佈——允許神經解碼器有效地執行重要採樣。結果的外觀模型支持各向異性採樣和級別渲染，並允許將深層材質圖層烘焙成緊湊統一的神經表示。通過將硬件加速的張量操作暴露給光線追踪著色器，我們展示了在實時路徑追踪器內有效地內聯和執行神經解碼器是可能的。我們分析了隨著神經材料數量的增加而增加的可擴展性，並建議使用針對一致和分歧執行進行優化的代碼來提高性能。我們的神經材料著色器可以比非神經分層材料快一個數量級。這為在遊戲和實時預覽等應用中使用電影質量的視覺效果敞開了大門。

English

We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior -- transformation of directions into learned shading frames -- facilitates accurate reconstruction of mesoscale effects. The second prior -- a microfacet sampling distribution -- allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.

即時神經外觀模型

Real-Time Neural Appearance Models

摘要

Support