PE3R：感知高效的三維重建

PE3R: Perception-Efficient 3D Reconstruction

March 10, 2025

作者: Jie Hu, Shizun Wang, Xinchao Wang

cs.AI

摘要

近期在二維到三維感知領域的進展顯著提升了從二維圖像理解三維場景的能力。然而，現有方法面臨著關鍵挑戰，包括跨場景的泛化能力有限、感知精度欠佳以及重建速度緩慢。為解決這些限制，我們提出了感知高效的三維重建框架（PE3R），這是一種旨在同時提升精度和效率的新穎框架。PE3R採用前饋架構，實現快速的三維語義場重建。該框架在跨多樣場景和物體的零樣本泛化上表現出強健性，同時顯著提高了重建速度。在二維到三維開放詞彙分割和三維重建上的大量實驗驗證了PE3R的有效性和多功能性。該框架在三維語義場重建中實現了至少9倍的速度提升，並在感知精度和重建精確度上取得了顯著進步，為該領域樹立了新的基準。代碼已公開於：https://github.com/hujiecpp/PE3R。

English

Recent advancements in 2D-to-3D perception have significantly improved the understanding of 3D scenes from 2D images. However, existing methods face critical challenges, including limited generalization across scenes, suboptimal perception accuracy, and slow reconstruction speeds. To address these limitations, we propose Perception-Efficient 3D Reconstruction (PE3R), a novel framework designed to enhance both accuracy and efficiency. PE3R employs a feed-forward architecture to enable rapid 3D semantic field reconstruction. The framework demonstrates robust zero-shot generalization across diverse scenes and objects while significantly improving reconstruction speed. Extensive experiments on 2D-to-3D open-vocabulary segmentation and 3D reconstruction validate the effectiveness and versatility of PE3R. The framework achieves a minimum 9-fold speedup in 3D semantic field reconstruction, along with substantial gains in perception accuracy and reconstruction precision, setting new benchmarks in the field. The code is publicly available at: https://github.com/hujiecpp/PE3R.