4DSloMo：基于异步捕捉的高速场景四维重建

摘要

從多視角視頻中重建快速動態場景對於高速運動分析及逼真的四維重建至關重要。然而，大多數四維捕捉系統的幀率限制在30 FPS（每秒幀數）以下，直接從低幀率輸入進行高速運動的四維重建可能導致不理想的結果。本研究提出了一種僅使用低幀率相機的高速四維捕捉系統，通過創新的捕捉與處理模塊實現。在捕捉方面，我們提出了一種異步捕捉方案，通過錯開相機的啟動時間來提高有效幀率。通過分組相機並利用25 FPS的基礎幀率，我們的方法在不需專用高速相機的情況下，達到了100至200 FPS的等效幀率。在處理方面，我們還提出了一種新穎的生成模型，以修復由四維稀疏視角重建引起的偽影，因為異步性減少了每個時間戳的視點數量。具體而言，我們建議訓練一個基於視頻擴散的偽影修復模型，用於稀疏四維重建，該模型能夠精細化缺失細節、保持時間一致性，並提升整體重建質量。實驗結果表明，與同步捕捉相比，我們的方法顯著增強了高速四維重建的效果。

English

Reconstructing fast-dynamic scenes from multi-view videos is crucial for high-speed motion analysis and realistic 4D reconstruction. However, the majority of 4D capture systems are limited to frame rates below 30 FPS (frames per second), and a direct 4D reconstruction of high-speed motion from low FPS input may lead to undesirable results. In this work, we propose a high-speed 4D capturing system only using low FPS cameras, through novel capturing and processing modules. On the capturing side, we propose an asynchronous capture scheme that increases the effective frame rate by staggering the start times of cameras. By grouping cameras and leveraging a base frame rate of 25 FPS, our method achieves an equivalent frame rate of 100-200 FPS without requiring specialized high-speed cameras. On processing side, we also propose a novel generative model to fix artifacts caused by 4D sparse-view reconstruction, as asynchrony reduces the number of viewpoints at each timestamp. Specifically, we propose to train a video-diffusion-based artifact-fix model for sparse 4D reconstruction, which refines missing details, maintains temporal consistency, and improves overall reconstruction quality. Experimental results demonstrate that our method significantly enhances high-speed 4D reconstruction compared to synchronous capture.

4DSloMo：基于异步捕捉的高速场景四维重建

4DSloMo: 4D Reconstruction for High Speed Scene with Asynchronous Capture

摘要

Support