Scal3R：面向可扩展在线三维重建的高效多相对位姿查询学习

摘要

在线三维重建模型在长视频上表现不佳。其原因在于，相对于固定的首帧锚点回归位姿，会迫使模型在远超训练分布的范围外进行外推。微小的漂移不断累积并放大，最终导致严重的几何坍塌。然而，我们观察到，在整个失效过程中，逐帧深度始终保持稳定。主干网络提取的局部几何信息依然完好，只有全局位姿头发生了崩溃。受这一解耦现象的启发，我们提出了Scal3R。该方法将在线重建重新表述为多参考帧相对位姿查询问题。我们使用轻量级可学习令牌（仅占参数总量的约1%），并通过非对称注意力将其注入完全冻结的主干网络中。这一机制使得模型能够相对于多个历史关键帧查询位姿。配合带闭环检测的在线位姿图优化系统，可有效抑制长期漂移。Scal3R在单块GPU上仅需8小时即可收敛。与在线基线方法相比，其在KITTI数据集上的平均ATE降低了超过60%。此外，该方法在Virtual KITTI、Sintel、TUM-Dynamic、ScanNet和7-Scenes等多个基准上均取得了最先进的性能。项目页面：https://linjohnss.github.io/scal3r/

English

Online 3D reconstruction models perform poorly on long videos. This happens because regressing poses relative to a fixed first-frame anchor forces extrapolation far beyond the training distribution. Small drifts accumulate and amplify into significant geometric collapse. However, we observe that per-frame depth remains stable throughout this failure. The backbone's local geometry remains intact; only the global pose head breaks down. Motivated by this decoupling, we introduce Scal3R. This approach reformulates online reconstruction as multi-reference relative pose querying. We use lightweight learnable tokens, which make up about ~1% of the parameters, and inject them into a completely frozen backbone via asymmetric attention. This setup queries poses relative to multiple past keyframes. An online pose-graph optimization system with loop closure suppresses long-range drift. Scal3R reaches convergence in 8 hours on a single GPU. It reduces the average ATE by over 60% on KITTI compared to the online baseline. It also achieves state-of-the-art performance across Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes. Project page: https://linjohnss.github.io/scal3r/