LiftFeat：基於3D幾何感知的局部特徵匹配

LiftFeat: 3D Geometry-Aware Local Feature Matching

May 6, 2025

作者: Yepeng Liu, Wenpeng Lai, Zhou Zhao, Yuxuan Xiong, Jinchi Zhu, Jun Cheng, Yongchao Xu

cs.AI

摘要

在機器人技術的同步定位與地圖構建（SLAM）及視覺定位等應用中，穩健且高效的局部特徵匹配扮演著至關重要的角色。儘管已取得顯著進展，在光照劇烈變化、低紋理區域或重複圖案等場景下，提取出既穩健又具區分性的視覺特徵仍是一大挑戰。本文提出了一種名為LiftFeat的新型輕量級網絡，該網絡通過聚合三維幾何特徵來提升原始描述符的魯棒性。具體而言，我們首先採用預訓練的單目深度估計模型生成偽表面法線標籤，以此監督基於預測表面法線的三維幾何特徵提取過程。隨後，我們設計了一個三維幾何感知的特徵提升模塊，將表面法線特徵與原始二維描述符特徵進行融合。這種三維幾何特徵的整合，增強了二維特徵描述在極端條件下的區分能力。在相對姿態估計、單應性估計及視覺定位任務上的大量實驗結果表明，我們的LiftFeat在性能上超越了一些輕量級的現有最先進方法。代碼將發佈於：https://github.com/lyp-deeplearning/LiftFeat。

English

Robust and efficient local feature matching plays a crucial role in applications such as SLAM and visual localization for robotics. Despite great progress, it is still very challenging to extract robust and discriminative visual features in scenarios with drastic lighting changes, low texture areas, or repetitive patterns. In this paper, we propose a new lightweight network called LiftFeat, which lifts the robustness of raw descriptor by aggregating 3D geometric feature. Specifically, we first adopt a pre-trained monocular depth estimation model to generate pseudo surface normal label, supervising the extraction of 3D geometric feature in terms of predicted surface normal. We then design a 3D geometry-aware feature lifting module to fuse surface normal feature with raw 2D descriptor feature. Integrating such 3D geometric feature enhances the discriminative ability of 2D feature description in extreme conditions. Extensive experimental results on relative pose estimation, homography estimation, and visual localization tasks, demonstrate that our LiftFeat outperforms some lightweight state-of-the-art methods. Code will be released at : https://github.com/lyp-deeplearning/LiftFeat.