Multi4D: 通过多级竞争分配实现高保真动态高斯泼溅

摘要

動態3D高斯潑濺在運動一致性與視覺保真度之間存在根本性的張力。基於變形的方法保留了時間對應關係，但卻因運動過度因子化而平滑了高頻動態；相對地，4D基元方法能捕捉精細視覺細節，卻因時間過度參數化而破壞物體身份，並導致嚴重的儲存開銷。為了解決此問題，我們提出Multi4D——一個基於多級競爭分配的高保真動態高斯潑濺框架。不同於單一表徵，我們將建模能力分散到三個結構化層級：靜態結構、持久動態幾何，以及瞬態外觀基元。透過共享光柵化與殘差驅動優化，這些層級動態競爭以解釋光度誤差，實現無需預先分配分解的自適應特化。此分配方式在保留長期運動一致性的同時，捕捉精細動態細節，並以顯著更少的動態基元達成最先進的渲染品質與即時性能。此外，由於我們的表徵會明確追蹤隨時間變化的緊湊持久高斯體，後續可嵌入語意特徵，使Multi4D在達到最先進4D分割準確度的同時，實現一個數量級的加速。專案頁面：https://batfacewayne.github.io/Multi4D.io/

English

Dynamic 3D Gaussian splatting faces a fundamental tension between motion consistency and visual fidelity. Deformation-based approaches preserve temporal correspondence but suffer from motion over-factorization, oversmoothing high-frequency dynamics. In contrast, 4D-primitive methods capture fine visual details yet incur temporal overparameterization, breaking object identity and leading to severe storage overhead. To resolve this, we introduce Multi4D, a framework for high-fidelity dynamic Gaussian Splatting based on multi-level competitive allocation. Instead of a monolithic representation, we distribute modeling capacity across three structured levels: static structure, persistent dynamic geometry, and transient appearance primitives. Through shared rasterization and residual-driven optimization, these levels dynamically compete to explain photometric error, enabling adaptive specialization without pre-assigned decomposition. This allocation preserves long-term motion consistency while capturing fine dynamic detail, achieving state-of-the-art rendering quality and real-time performance with significantly fewer dynamic primitives. Furthermore, because our representation explicitly tracks compact persistent Gaussians over time, semantic features can be embedded afterward, enabling Multi4D to achieve state-of-the-art 4D segmentation accuracy with an order-of-magnitude speedup. Project page: https://batfacewayne.github.io/Multi4D.io/