SpanCalib-VLM: 校准的视觉-语言模型幻觉跨度检测

摘要

在大规模视觉-语言模型（LVLMs）中检测幻觉，既需要精确的片段定位，也需要良好校准的置信度分数。微调后的生成式视觉-语言模型擅长识别幻觉文本片段，但存在过度自信和推理延迟高的问题。判别式序列标注器具有确定性的速度和优越的校准性能，但片段召回率较为保守。我们提出了SpanCalib-VLM，一种用于SHROOM-Visions共享任务的混合双系统，它将多模态序列标注器（由XLM-RoBERTa-Large通过交叉注意力与SigLIP视觉编码器融合而成）与我们的微调生成式VLM（Qwen3.5-4B-SHROOM-SFT）相结合。通过联合校准融合策略，生成模型产生的候选片段会被序列标注器提供的校准概率重新评分。在SHROOM-Visions英文评估集上，我们的集成系统取得了0.41的皮尔逊校准相关系数、0.39的总体IoU，其中干净响应IoU为0.91}，总体检测准确率为70.7%。我们公开提供模型权重和代码。

English

Detecting hallucinations in Large Vision-Language Models (LVLMs) requires both accurate span localization and well-calibrated confidence scores. Fine-tuned generative VLMs excel at identifying hallucinated text spans but suffer from overconfidence and high inference latency. Discriminative sequence taggers offer deterministic speed and superior calibration but exhibit conservative span recall. We present SpanCalib-VLM, a hybrid dual-system for the SHROOM-Visions Shared Task that combines a multimodal sequence tagger, consisting of XLM-RoBERTa-Large fused with a SigLIP vision encoder via cross-attention, with our fine-tuned generative VLM (Qwen3.5-4B-SHROOM-SFT). Through a Union-Calibrated Fusion strategy, candidate spans from the generative model are re-scored with calibrated probabilities from the sequence tagger. On the SHROOM-Visions English evaluation split, our ensemble achieves a Pearson calibration correlation of 0.41 and an overall IoU of 0.39, with a clean-response IoU of 0.91} and overall detection accuracy of 70.7%. We make our model weights and code publicly available.