ChatPaper.aiChatPaper

高斯图像:通过二维高斯喷洒实现的1000 FPS图像表示和压缩

GaussianImage: 1000 FPS Image Representation and Compression by 2D Gaussian Splatting

March 13, 2024
作者: Xinjie Zhang, Xingtong Ge, Tongda Xu, Dailan He, Yan Wang, Hongwei Qin, Guo Lu, Jing Geng, Jun Zhang
cs.AI

摘要

最近,隐式神经表示(INRs)在图像表示和压缩方面取得了巨大成功,提供了高视觉质量和快速渲染速度,可达10-1000 FPS,假定有足够的GPU资源可用。然而,这一要求通常阻碍了它们在内存有限的低端设备上的使用。为此,我们提出了一种通过二维高斯飞溅(2D Gaussian Splatting)进行图像表示和压缩的开创性范式,命名为GaussianImage。我们首先引入二维高斯来表示图像,其中每个高斯具有8个参数,包括位置、协方差和颜色。随后,我们揭示了一种基于累积求和的新型渲染算法。值得注意的是,我们的方法在GPU内存使用方面至少降低了3倍,拟合时间快了5倍,不仅在表示性能上与INRs(例如WIRE、I-NGP)不相上下,而且无论参数大小如何,渲染速度都达到了1500-2000 FPS。此外,我们还整合了现有的矢量量化技术来构建图像编解码器。实验结果表明,我们的编解码器在速率失真性能上与基于压缩的INRs(如COIN和COIN++)相媲美,同时实现了约1000 FPS的解码速度。此外,初步概念验证表明,我们的编解码器在使用部分比特回传编码时超越了COIN和COIN++的性能。
English
Implicit neural representations (INRs) recently achieved great success in image representation and compression, offering high visual quality and fast rendering speeds with 10-1000 FPS, assuming sufficient GPU resources are available. However, this requirement often hinders their use on low-end devices with limited memory. In response, we propose a groundbreaking paradigm of image representation and compression by 2D Gaussian Splatting, named GaussianImage. We first introduce 2D Gaussian to represent the image, where each Gaussian has 8 parameters including position, covariance and color. Subsequently, we unveil a novel rendering algorithm based on accumulated summation. Remarkably, our method with a minimum of 3times lower GPU memory usage and 5times faster fitting time not only rivals INRs (e.g., WIRE, I-NGP) in representation performance, but also delivers a faster rendering speed of 1500-2000 FPS regardless of parameter size. Furthermore, we integrate existing vector quantization technique to build an image codec. Experimental results demonstrate that our codec attains rate-distortion performance comparable to compression-based INRs such as COIN and COIN++, while facilitating decoding speeds of approximately 1000 FPS. Additionally, preliminary proof of concept shows that our codec surpasses COIN and COIN++ in performance when using partial bits-back coding.

Summary

AI-Generated Summary

PDF104December 15, 2024