預訓練模型時代下的非擺拍稀疏視角房間佈局重建

Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Model

February 24, 2025

作者: Yaxuan Huang, Xili Dai, Jianan Wang, Xianbiao Qi, Yixing Yuan, Xiangyu Yue

cs.AI

摘要

從多視角圖像進行房間佈局估計的研究尚不充分，這主要源於多視角幾何帶來的複雜性，需要多步驟解決方案，如相機內外參數估計、圖像匹配和三角測量。然而，在三維重建領域，近期三維基礎模型（如DUSt3R）的發展，已將傳統的多步驟運動結構恢復過程轉變為端到端的單步方法。基於此，我們提出了Plane-DUSt3R，這是一種利用三維基礎模型DUSt3R進行多視角房間佈局估計的新方法。Plane-DUSt3R整合了DUSt3R框架，並在房間佈局數據集（Structure3D）上進行微調，調整目標以估計結構平面。通過生成均勻且簡潔的結果，Plane-DUSt3R僅需單一後處理步驟和二維檢測結果即可完成房間佈局估計。與以往依賴單視角或全景圖像的方法不同，Plane-DUSt3R擴展了處理多視角圖像的設定。此外，它提供了一個簡化的端到端解決方案，簡化了流程並減少了誤差累積。實驗結果表明，Plane-DUSt3R不僅在合成數據集上超越了現有最先進的方法，而且在包含不同圖像風格（如卡通）的真實數據上也展現出魯棒性和有效性。我們的代碼可在以下網址獲取：https://github.com/justacar/Plane-DUSt3R。

English

Room layout estimation from multiple-perspective images is poorly investigated due to the complexities that emerge from multi-view geometry, which requires muti-step solutions such as camera intrinsic and extrinsic estimation, image matching, and triangulation. However, in 3D reconstruction, the advancement of recent 3D foundation models such as DUSt3R has shifted the paradigm from the traditional multi-step structure-from-motion process to an end-to-end single-step approach. To this end, we introduce Plane-DUSt3R, a novel method for multi-view room layout estimation leveraging the 3D foundation model DUSt3R. Plane-DUSt3R incorporates the DUSt3R framework and fine-tunes on a room layout dataset (Structure3D) with a modified objective to estimate structural planes. By generating uniform and parsimonious results, Plane-DUSt3R enables room layout estimation with only a single post-processing step and 2D detection results. Unlike previous methods that rely on single-perspective or panorama image, Plane-DUSt3R extends the setting to handle multiple-perspective images. Moreover, it offers a streamlined, end-to-end solution that simplifies the process and reduces error accumulation. Experimental results demonstrate that Plane-DUSt3R not only outperforms state-of-the-art methods on the synthetic dataset but also proves robust and effective on in the wild data with different image styles such as cartoon.Our code is available at: https://github.com/justacar/Plane-DUSt3R