StreamSplat：面向未校准视频流的在线动态三维重建

摘要

從未校準的視頻流中實時重建動態三維場景，對於眾多實際應用至關重要。然而，現有方法難以同時應對三大挑戰：1）實時處理未校準的輸入數據，2）精確建模動態場景的演變，以及3）保持長期的穩定性和計算效率。為此，我們提出了StreamSplat，這是首個完全前饋的框架，能夠以在線方式將任意長度的未校準視頻流轉化為動態三維高斯潑濺（3DGS）表示，並能從時間局部觀測中恢復場景動態。我們提出了兩項關鍵技術創新：在3DGS位置預測的靜態編碼器中引入概率採樣機制，以及在動態解碼器中採用雙向變形場，從而實現了魯棒且高效的動態建模。在靜態與動態基準測試上的廣泛實驗表明，StreamSplat在重建質量和動態場景建模方面均持續超越先前工作，同時獨特地支持任意長度視頻流的在線重建。代碼和模型可通過https://github.com/nickwzk/StreamSplat獲取。

English

Real-time reconstruction of dynamic 3D scenes from uncalibrated video streams is crucial for numerous real-world applications. However, existing methods struggle to jointly address three key challenges: 1) processing uncalibrated inputs in real time, 2) accurately modeling dynamic scene evolution, and 3) maintaining long-term stability and computational efficiency. To this end, we introduce StreamSplat, the first fully feed-forward framework that transforms uncalibrated video streams of arbitrary length into dynamic 3D Gaussian Splatting (3DGS) representations in an online manner, capable of recovering scene dynamics from temporally local observations. We propose two key technical innovations: a probabilistic sampling mechanism in the static encoder for 3DGS position prediction, and a bidirectional deformation field in the dynamic decoder that enables robust and efficient dynamic modeling. Extensive experiments on static and dynamic benchmarks demonstrate that StreamSplat consistently outperforms prior works in both reconstruction quality and dynamic scene modeling, while uniquely supporting online reconstruction of arbitrarily long video streams. Code and models are available at https://github.com/nickwzk/StreamSplat.

StreamSplat：面向未校准视频流的在线动态三维重建

StreamSplat: Towards Online Dynamic 3D Reconstruction from Uncalibrated Video Streams

摘要

Support