場景座標重建：通過重新定位器的增量學習對圖像集合進行姿態設定

Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizer

April 22, 2024

作者: Eric Brachmann, Jamie Wynn, Shuai Chen, Tommaso Cavallari, Áron Monszpart, Daniyar Turmukhambetov, Victor Adrian Prisacariu

cs.AI

摘要

我們致力於從描繪場景的一組影像中估計相機參數的任務。流行的基於特徵的結構從運動（SfM）工具通過增量重建來解決這個任務：它們重複三角測量稀疏的3D點並將更多相機視圖註冊到稀疏點雲。我們重新解釋增量式結構從運動為一個視覺重新定位器的迭代應用和細化，即一種將新視圖註冊到重建當前狀態的方法。這種觀點使我們能夠研究不基於局部特徵匹配的替代視覺重新定位器。我們展示了一種稱為場景坐標回歸的基於學習的重新定位方法，允許我們從未定位的影像中構建隱式的神經場景表示。與其他基於學習的重建方法不同，我們不需要姿態先驗知識或順序輸入，並且我們可以高效地優化數千張影像。我們的方法，ACE0（ACE Zero），通過新穎視圖合成展示了與基於特徵的SfM相當的相機姿勢估計精度。項目頁面：https://nianticlabs.github.io/acezero/

English

We address the task of estimating camera parameters from a set of images depicting a scene. Popular feature-based structure-from-motion (SfM) tools solve this task by incremental reconstruction: they repeat triangulation of sparse 3D points and registration of more camera views to the sparse point cloud. We re-interpret incremental structure-from-motion as an iterated application and refinement of a visual relocalizer, that is, of a method that registers new views to the current state of the reconstruction. This perspective allows us to investigate alternative visual relocalizers that are not rooted in local feature matching. We show that scene coordinate regression, a learning-based relocalization approach, allows us to build implicit, neural scene representations from unposed images. Different from other learning-based reconstruction methods, we do not require pose priors nor sequential inputs, and we optimize efficiently over thousands of images. Our method, ACE0 (ACE Zero), estimates camera poses to an accuracy comparable to feature-based SfM, as demonstrated by novel view synthesis. Project page: https://nianticlabs.github.io/acezero/