HiFi4G：通过紧凑的高斯飞溅实现高保真人类表现渲染

摘要

我们最近在照片级逼真人体建模和渲染方面取得了巨大进展。然而，高效地渲染逼真的人体表现并将其整合到光栅化流程中仍然具有挑战性。在本文中，我们提出了HiFi4G，这是一种明确而紧凑的基于高斯的方法，用于从密集镜头素材中渲染高保真度的人体表现。我们的核心思想是将3D高斯表示与非刚性跟踪相结合，实现紧凑且适合压缩的表示。我们首先提出了一个双图机制来获取运动先验，使用粗略变形图进行有效初始化，并使用细粒度高斯图来强制执行后续约束。然后，我们利用具有自适应时空正则化器的4D高斯优化方案，有效平衡非刚性先验和高斯更新。我们还提出了一个伴随的压缩方案，通过残差补偿在各种平台上实现沉浸式体验。它实现了约25倍的大幅压缩率，每帧存储不到2MB。大量实验证明了我们方法的有效性，在优化速度、渲染质量和存储开销方面明显优于现有方法。

English

We have recently seen tremendous progress in photo-real human modeling and rendering. Yet, efficiently rendering realistic human performance and integrating it into the rasterization pipeline remains challenging. In this paper, we present HiFi4G, an explicit and compact Gaussian-based approach for high-fidelity human performance rendering from dense footage. Our core intuition is to marry the 3D Gaussian representation with non-rigid tracking, achieving a compact and compression-friendly representation. We first propose a dual-graph mechanism to obtain motion priors, with a coarse deformation graph for effective initialization and a fine-grained Gaussian graph to enforce subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to effectively balance the non-rigid prior and Gaussian updating. We also present a companion compression scheme with residual compensation for immersive experiences on various platforms. It achieves a substantial compression rate of approximately 25 times, with less than 2MB of storage per frame. Extensive experiments demonstrate the effectiveness of our approach, which significantly outperforms existing approaches in terms of optimization speed, rendering quality, and storage overhead.

HiFi4G：通过紧凑的高斯飞溅实现高保真人类表现渲染

HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting

摘要

Support