HiFi4G：透過緊湊的高斯濺射實現高保真人類表現呈現

摘要

最近，在照片逼真的人類建模和渲染方面取得了巨大的進展。然而，高效地渲染逼真的人類表現並將其整合到光柵化流程中仍然具有挑戰性。在本文中，我們提出了HiFi4G，一種明確且緊湊的基於高斯的方法，用於從密集影片中渲染高保真的人類表現。我們的核心想法是將3D高斯表示法與非剛性跟踪結合，實現緊湊且適合壓縮的表示法。我們首先提出了一種雙圖機制來獲得運動先驗，使用粗略變形圖進行有效初始化，並使用精細的高斯圖來強制實施後續約束。然後，我們利用一種具有自適應時空正則化器的4D高斯優化方案，有效平衡非剛性先驗和高斯更新。我們還提出了一種伴隨的壓縮方案，通過殘差補償實現在各種平台上的身臨其境體驗。它實現了約25倍的顯著壓縮率，每幀存儲不到2MB。大量實驗證明了我們方法的有效性，在優化速度、渲染質量和存儲開銷方面明顯優於現有方法。

English

We have recently seen tremendous progress in photo-real human modeling and rendering. Yet, efficiently rendering realistic human performance and integrating it into the rasterization pipeline remains challenging. In this paper, we present HiFi4G, an explicit and compact Gaussian-based approach for high-fidelity human performance rendering from dense footage. Our core intuition is to marry the 3D Gaussian representation with non-rigid tracking, achieving a compact and compression-friendly representation. We first propose a dual-graph mechanism to obtain motion priors, with a coarse deformation graph for effective initialization and a fine-grained Gaussian graph to enforce subsequent constraints. Then, we utilize a 4D Gaussian optimization scheme with adaptive spatial-temporal regularizers to effectively balance the non-rigid prior and Gaussian updating. We also present a companion compression scheme with residual compensation for immersive experiences on various platforms. It achieves a substantial compression rate of approximately 25 times, with less than 2MB of storage per frame. Extensive experiments demonstrate the effectiveness of our approach, which significantly outperforms existing approaches in terms of optimization speed, rendering quality, and storage overhead.

HiFi4G：透過緊湊的高斯濺射實現高保真人類表現呈現

HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting

摘要

Support