三维目标检测与视觉-语言模型综述

摘要

本综述系统分析了视觉-语言模型（VLMs）在三维物体检测领域的全面研究进展，这一领域正处于三维视觉与多模态人工智能的交叉前沿。通过审视超过100篇研究论文，我们首次提供了专门针对视觉-语言模型在三维物体检测中的系统性分析。首先，我们概述了视觉-语言模型在三维物体检测中面临的独特挑战，特别是在空间推理和数据复杂性方面与二维检测的区别。传统方法如点云和体素网格被与现代视觉-语言框架如CLIP和三维大语言模型（3D LLMs）进行对比，后者支持开放词汇检测和零样本泛化。我们回顾了关键架构、预训练策略以及提示工程方法，这些方法通过文本与三维特征的对齐，有效实现了基于视觉-语言模型的三维物体检测。通过可视化示例和评估基准的讨论，展示了其性能与行为特征。最后，我们指出了当前面临的挑战，如三维-语言数据集的局限性和计算需求，并提出了推动视觉-语言模型在三维物体检测中发展的未来研究方向。>物体检测，视觉-语言模型，智能体，VLMs，LLMs，人工智能

English

This review provides a systematic analysis of comprehensive survey of 3D object detection with vision-language models(VLMs) , a rapidly advancing area at the intersection of 3D vision and multimodal AI. By examining over 100 research papers, we provide the first systematic analysis dedicated to 3D object detection with vision-language models. We begin by outlining the unique challenges of 3D object detection with vision-language models, emphasizing differences from 2D detection in spatial reasoning and data complexity. Traditional approaches using point clouds and voxel grids are compared to modern vision-language frameworks like CLIP and 3D LLMs, which enable open-vocabulary detection and zero-shot generalization. We review key architectures, pretraining strategies, and prompt engineering methods that align textual and 3D features for effective 3D object detection with vision-language models. Visualization examples and evaluation benchmarks are discussed to illustrate performance and behavior. Finally, we highlight current challenges, such as limited 3D-language datasets and computational demands, and propose future research directions to advance 3D object detection with vision-language models. >Object Detection, Vision-Language Models, Agents, VLMs, LLMs, AI

三维目标检测与视觉-语言模型综述

A Review of 3D Object Detection with Vision-Language Models

摘要

Summary

Support

Support