GSTAR：高斯表面追踪與重建

GSTAR: Gaussian Surface Tracking and Reconstruction

January 17, 2025

作者: Chengwei Zheng, Lixin Xue, Juan Zarate, Jie Song

cs.AI

摘要

3D 高斯點陣技術已實現靜態場景的高效照片逼真渲染。最近的研究將這些方法擴展到支持表面重建和跟踪。然而，使用 3D 高斯函數跟踪動態表面仍然具有挑戰性，因為存在著複雜的拓撲變化，例如表面的出現、消失或分裂。為應對這些挑戰，我們提出了一種新方法 GSTAR，實現了對於具有變化拓撲的一般動態場景的照片逼真渲染、準確表面重建和可靠的 3D 跟踪。給定多視圖捕獲作為輸入，GSTAR 將高斯函數綁定到網格面以表示動態物體。對於具有一致拓撲的表面，GSTAR 保持網格拓撲並使用高斯函數跟踪網格。在拓撲變化的區域，GSTAR 自適應地從網格解除高斯函數的綁定，實現準確的配准並基於這些優化的高斯函數生成新表面。此外，我們引入了一種基於表面的場景流方法，為幀間跟踪提供了堅固的初始化。實驗證明我們的方法有效地跟踪和重建動態表面，實現了一系列應用。我們的項目頁面及代碼發布可在 https://eth-ait.github.io/GSTAR/ 找到。

English

3D Gaussian Splatting techniques have enabled efficient photo-realistic rendering of static scenes. Recent works have extended these approaches to support surface reconstruction and tracking. However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians. In regions where topology changes, GSTAR adaptively unbinds Gaussians from the mesh, enabling accurate registration and the generation of new surfaces based on these optimized Gaussians. Additionally, we introduce a surface-based scene flow method that provides robust initialization for tracking between frames. Experiments demonstrate that our method effectively tracks and reconstructs dynamic surfaces, enabling a range of applications. Our project page with the code release is available at https://eth-ait.github.io/GSTAR/.