Multi4D: 通过多级竞争分配的高保真度动态高斯泼溅

摘要

动态三维高斯溅射在运动一致性与视觉保真度之间面临根本性矛盾。基于形变的方法能够保持时序对应关系，但会因运动过度分解而平滑高频动态；而四维基元方法虽能捕捉精细视觉细节，却因时序过度参数化导致物体身份断裂，并带来严重存储开销。为解决这一问题，我们提出Multi4D——一种基于多层次竞争分配的高保真动态高斯溅射框架。不同于单一表示，我们将建模能力分布到三个结构化层次：静态结构、持久动态几何与瞬态外观基元。通过共享光栅化与残差驱动优化，这些层次动态竞争解释光度误差，无需预先分配即可实现自适应专业化。这种分配机制在保留长期运动一致性的同时捕捉精细动态细节，以显著更少的动态基元实现最先进的渲染质量与实时性能。此外，由于我们的表示显式追踪随时间变化的紧凑持久高斯体，语义特征可随后嵌入，使Multi4D以数量级加速实现最先进的四维分割精度。项目页面：https://batfacewayne.github.io/Multi4D.io/

English

Dynamic 3D Gaussian splatting faces a fundamental tension between motion consistency and visual fidelity. Deformation-based approaches preserve temporal correspondence but suffer from motion over-factorization, oversmoothing high-frequency dynamics. In contrast, 4D-primitive methods capture fine visual details yet incur temporal overparameterization, breaking object identity and leading to severe storage overhead. To resolve this, we introduce Multi4D, a framework for high-fidelity dynamic Gaussian Splatting based on multi-level competitive allocation. Instead of a monolithic representation, we distribute modeling capacity across three structured levels: static structure, persistent dynamic geometry, and transient appearance primitives. Through shared rasterization and residual-driven optimization, these levels dynamically compete to explain photometric error, enabling adaptive specialization without pre-assigned decomposition. This allocation preserves long-term motion consistency while capturing fine dynamic detail, achieving state-of-the-art rendering quality and real-time performance with significantly fewer dynamic primitives. Furthermore, because our representation explicitly tracks compact persistent Gaussians over time, semantic features can be embedded afterward, enabling Multi4D to achieve state-of-the-art 4D segmentation accuracy with an order-of-magnitude speedup. Project page: https://batfacewayne.github.io/Multi4D.io/