1000+ FPS 动态场景渲染的4D高斯泼溅技术

1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering

March 20, 2025

作者: Yuheng Yuan, Qiuhong Shen, Xingyi Yang, Xinchao Wang

cs.AI

摘要

4D高斯泼溅（4DGS）作为一种动态场景重建方法，近期获得了广泛关注。尽管其重建质量卓越，但4DGS通常需要大量存储空间且渲染速度较慢。本研究深入探讨了这些问题，并识别出两个关键的时间冗余来源：（Q1）短生命周期高斯：4DGS使用大量时间跨度短的高斯函数来表现场景动态，导致高斯函数数量过多。（Q2）非活跃高斯：在渲染时，每一帧仅有一小部分高斯函数实际贡献于画面，然而在光栅化过程中所有高斯函数均被处理，造成了计算冗余。针对这些冗余，我们提出了4DGS-1K，该方案在现代GPU上运行速度超过1000 FPS。针对Q1，我们引入了时空变化评分这一新的剪枝准则，有效剔除短生命周期高斯，同时促使4DGS利用时间跨度更长的高斯函数捕捉场景动态。针对Q2，我们为连续帧中的活跃高斯函数存储掩码，显著减少了渲染中的冗余计算。与原始4DGS相比，我们的方法在复杂动态场景上实现了存储空间减少41倍、光栅化速度提升9倍，同时保持了相当的视觉质量。详情请访问我们的项目页面：https://4DGS-1K.github.io。

English

4D Gaussian Splatting (4DGS) has recently gained considerable attention as a method for reconstructing dynamic scenes. Despite achieving superior quality, 4DGS typically requires substantial storage and suffers from slow rendering speed. In this work, we delve into these issues and identify two key sources of temporal redundancy. (Q1) Short-Lifespan Gaussians: 4DGS uses a large portion of Gaussians with short temporal span to represent scene dynamics, leading to an excessive number of Gaussians. (Q2) Inactive Gaussians: When rendering, only a small subset of Gaussians contributes to each frame. Despite this, all Gaussians are processed during rasterization, resulting in redundant computation overhead. To address these redundancies, we present 4DGS-1K, which runs at over 1000 FPS on modern GPUs. For Q1, we introduce the Spatial-Temporal Variation Score, a new pruning criterion that effectively removes short-lifespan Gaussians while encouraging 4DGS to capture scene dynamics using Gaussians with longer temporal spans. For Q2, we store a mask for active Gaussians across consecutive frames, significantly reducing redundant computations in rendering. Compared to vanilla 4DGS, our method achieves a 41times reduction in storage and 9times faster rasterization speed on complex dynamic scenes, while maintaining comparable visual quality. Please see our project page at https://4DGS-1K.github.io.