視覺語言模型在3D物體檢測中的應用綜述

摘要

本綜述系統性地分析了基於視覺-語言模型（VLMs）的三維物體檢測這一快速發展的領域，該領域位於三維視覺與多模態人工智慧的交叉點。通過審閱超過100篇研究論文，我們首次提供了專門針對視覺-語言模型的三維物體檢測的系統性分析。我們首先概述了視覺-語言模型在三維物體檢測中的獨特挑戰，強調了其在空間推理和數據複雜性方面與二維檢測的差異。傳統方法如點雲和體素網格與現代視覺-語言框架如CLIP和三維大語言模型（3D LLMs）進行了對比，後者支持開放詞彙檢測和零樣本泛化。我們回顧了關鍵的架構、預訓練策略以及提示工程方法，這些方法通過對齊文本和三維特徵來實現有效的基於視覺-語言模型的三維物體檢測。討論了可視化示例和評估基準，以展示其性能和行為。最後，我們指出了當前面臨的挑戰，如有限的三維-語言數據集和計算需求，並提出了未來研究方向，以推動基於視覺-語言模型的三維物體檢測的發展。>物體檢測，視覺-語言模型，智能體，VLMs，LLMs，人工智慧

English

This review provides a systematic analysis of comprehensive survey of 3D object detection with vision-language models(VLMs) , a rapidly advancing area at the intersection of 3D vision and multimodal AI. By examining over 100 research papers, we provide the first systematic analysis dedicated to 3D object detection with vision-language models. We begin by outlining the unique challenges of 3D object detection with vision-language models, emphasizing differences from 2D detection in spatial reasoning and data complexity. Traditional approaches using point clouds and voxel grids are compared to modern vision-language frameworks like CLIP and 3D LLMs, which enable open-vocabulary detection and zero-shot generalization. We review key architectures, pretraining strategies, and prompt engineering methods that align textual and 3D features for effective 3D object detection with vision-language models. Visualization examples and evaluation benchmarks are discussed to illustrate performance and behavior. Finally, we highlight current challenges, such as limited 3D-language datasets and computational demands, and propose future research directions to advance 3D object detection with vision-language models. >Object Detection, Vision-Language Models, Agents, VLMs, LLMs, AI

視覺語言模型在3D物體檢測中的應用綜述

A Review of 3D Object Detection with Vision-Language Models

摘要

Summary

Support

Support