高斯影像:透過2D高斯點陣投影實現每秒1000幀的影像表示和壓縮。
GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting
March 13, 2024
作者: Xinjie Zhang, Xingtong Ge, Tongda Xu, Dailan He, Yan Wang, Hongwei Qin, Guo Lu, Jing Geng, Jun Zhang
cs.AI
摘要
最近,隱式神經表示(INRs)在圖像表示和壓縮方面取得了巨大成功,提供高視覺品質和快速渲染速度,達到每秒10-1000幀,假設有足夠的GPU資源可用。然而,這種要求通常阻礙了它們在內存有限的低端設備上的使用。為此,我們提出了一種名為GaussianImage的基於2D高斯擴散的圖像表示和壓縮的開創性範式。我們首先引入2D高斯來表示圖像,其中每個高斯具有8個參數,包括位置、協方差和顏色。隨後,我們揭示了一種基於累積求和的新型渲染算法。值得注意的是,我們的方法在GPU內存使用量至少低3倍且擬合時間快5倍,不僅在表示性能上與INRs(例如WIRE、I-NGP)不相上下,而且無論參數大小如何,渲染速度都達到每秒1500-2000幀。此外,我們整合現有的向量量化技術來構建圖像編解碼器。實驗結果表明,我們的編解碼器在速率失真性能上與基於壓縮的INRs(如COIN和COIN++)相當,同時實現了約每秒1000幀的解碼速度。此外,初步概念證明,我們的編解碼器在使用部分bits-back編碼時優於COIN和COIN++的性能。
English
Implicit neural representations (INRs) recently achieved great success in
image representation and compression, offering high visual quality and fast
rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are
available. However, this requirement often hinders their use on low-end devices
with limited memory. In response, we propose a groundbreaking paradigm of image
representation and compression by 2D Gaussian Splatting, named GaussianImage.
We first introduce 2D Gaussian to represent the image, where each Gaussian has
8 parameters including position, covariance and color. Subsequently, we unveil
a novel rendering algorithm based on accumulated summation. Remarkably, our
method with a minimum of 3times lower GPU memory usage and 5times faster
fitting time not only rivals INRs (e.g., WIRE, I-NGP) in representation
performance, but also delivers a faster rendering speed of 1500-2000 FPS
regardless of parameter size. Furthermore, we integrate existing vector
quantization technique to build an image codec. Experimental results
demonstrate that our codec attains rate-distortion performance comparable to
compression-based INRs such as COIN and COIN++, while facilitating decoding
speeds of approximately 1000 FPS. Additionally, preliminary proof of concept
shows that our codec surpasses COIN and COIN++ in performance when using
partial bits-back coding.Summary
AI-Generated Summary