TRIPS：用於實時輻射場渲染的三線性點塗擴散

摘要

基於點的輝度場渲染在新視角合成方面展現了令人印象深刻的成果，提供了渲染品質和計算效率的極具吸引力的組合。然而，在這個領域中最新的方法也並非沒有缺點。3D 高斯濺射 [Kerbl 和 Kopanas 等人，2023] 在渲染高度詳細場景時遇到困難，因為會出現模糊和雲狀異常。另一方面，ADOP [Rückert 等人，2022] 能夠呈現更清晰的圖像，但神經重建網絡降低了性能，它面臨著時間不穩定性的挑戰，並且無法有效地處理點雲中的大間隙。在本文中，我們提出了 TRIPS（三線性點濺射），這是一種結合了高斯濺射和 ADOP 的思想的方法。我們新技術的基本概念涉及將點光柵化為屏幕空間圖像金字塔，金字塔層的選擇由投影點大小確定。這種方法允許使用單一三線性寫入渲染任意大的點。然後使用輕量級神經網絡重建一個無洞的圖像，其中包括超出濺射分辨率的細節。重要的是，我們的渲染管線是完全可微的，可以自動優化點的大小和位置。我們的評估表明，TRIPS 在渲染品質方面超越了現有的最先進方法，同時在現有硬件上實現了每秒 60 幀的實時幀速率。這種性能適用於具有複雜幾何、廣闊景觀和自動曝光素材等具有挑戰性的場景。

English

Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.

TRIPS：用於實時輻射場渲染的三線性點塗擴散

TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

摘要

Support