StreamSplat：面向未校准视频流的在线动态三维重建

StreamSplat: Towards Online Dynamic 3D Reconstruction from Uncalibrated Video Streams

June 10, 2025

作者: Zike Wu, Qi Yan, Xuanyu Yi, Lele Wang, Renjie Liao

cs.AI

摘要

从未校准视频流中实时重建动态3D场景对于众多实际应用至关重要。然而，现有方法难以同时应对三大关键挑战：1）实时处理未校准输入，2）精确建模动态场景演变，3）保持长期稳定性和计算效率。为此，我们提出了StreamSplat，这是首个完全前馈的框架，能够在线方式将任意长度的未校准视频流转换为动态3D高斯泼溅（3DGS）表示，并能从时间局部观测中恢复场景动态。我们提出了两项关键技术革新：在静态编码器中用于3DGS位置预测的概率采样机制，以及在动态解码器中实现稳健高效动态建模的双向变形场。在静态和动态基准上的大量实验表明，StreamSplat在重建质量和动态场景建模方面均优于先前工作，同时独特地支持任意长度视频流的在线重建。代码和模型可在https://github.com/nickwzk/StreamSplat获取。

English

Real-time reconstruction of dynamic 3D scenes from uncalibrated video streams is crucial for numerous real-world applications. However, existing methods struggle to jointly address three key challenges: 1) processing uncalibrated inputs in real time, 2) accurately modeling dynamic scene evolution, and 3) maintaining long-term stability and computational efficiency. To this end, we introduce StreamSplat, the first fully feed-forward framework that transforms uncalibrated video streams of arbitrary length into dynamic 3D Gaussian Splatting (3DGS) representations in an online manner, capable of recovering scene dynamics from temporally local observations. We propose two key technical innovations: a probabilistic sampling mechanism in the static encoder for 3DGS position prediction, and a bidirectional deformation field in the dynamic decoder that enables robust and efficient dynamic modeling. Extensive experiments on static and dynamic benchmarks demonstrate that StreamSplat consistently outperforms prior works in both reconstruction quality and dynamic scene modeling, while uniquely supporting online reconstruction of arbitrarily long video streams. Code and models are available at https://github.com/nickwzk/StreamSplat.