SpanCalib-VLM：視覺語言模型中校準的幻覺片段偵測

摘要

大型視覺-語言模型（LVLMs）中偵測幻覺，需要精確的片段定位與良好校準的置信度分數。微調後的生成式視覺-語言模型擅於識別幻覺文本片段，但存在過度自信與高推論延遲的問題。判別式序列標註器提供確定性的速度與優異的校準，但片段召回率較為保守。我們提出 SpanCalib-VLM，這是一個用於 SHROOM-Visions 共享任務的混合雙系統，結合了多模態序列標註器（由 XLM-RoBERTa-Large 與 SigLIP 視覺編碼器透過交叉注意力融合而成）與我們微調的生成式視覺-語言模型（Qwen3.5-4B-SHROOM-SFT）。透過聯合校準融合策略，來自生成式模型的候選片段會使用序列標註器的校準機率重新評分。在 SHROOM-Visions 英文評測分割上，我們的集成模型達到了 0.41 的皮爾遜校準相關係數與 0.39 的整體 IoU，其中潔淨回應的 IoU 為 0.91，整體偵測準確率為 70.7%。我們公開提供模型權重與程式碼。

English

Detecting hallucinations in Large Vision-Language Models (LVLMs) requires both accurate span localization and well-calibrated confidence scores. Fine-tuned generative VLMs excel at identifying hallucinated text spans but suffer from overconfidence and high inference latency. Discriminative sequence taggers offer deterministic speed and superior calibration but exhibit conservative span recall. We present SpanCalib-VLM, a hybrid dual-system for the SHROOM-Visions Shared Task that combines a multimodal sequence tagger, consisting of XLM-RoBERTa-Large fused with a SigLIP vision encoder via cross-attention, with our fine-tuned generative VLM (Qwen3.5-4B-SHROOM-SFT). Through a Union-Calibrated Fusion strategy, candidate spans from the generative model are re-scored with calibrated probabilities from the sequence tagger. On the SHROOM-Visions English evaluation split, our ensemble achieves a Pearson calibration correlation of 0.41 and an overall IoU of 0.39, with a clean-response IoU of 0.91} and overall detection accuracy of 70.7%. We make our model weights and code publicly available.