看见并非共享：一些视觉语言模型在不对称对话中高估共同基础

摘要

在协作对话中，共同感知并不能保证共同解释。相互理解必须通过交互建立。我们研究了视觉语言模型（VLM）能否通过共同基础建立，区分对话参与者之间“可能共享”的信息与“已经共享”的信息。我们将此问题形式化为一个解释匹配任务，基于来自HCRC MapTask对话的13,077条带注释的指代表达式，并在系统性地控制对话上下文和地图信息访问的条件下评估VLM。结果显示，提供真实地图图像虽能提升整体性能，但会导致模型过度预测对齐倾向。相同地图内容的文本描述重现了该偏差，而无关信息的图像则完全抑制了对齐预测，表明该偏差源于与任务相关的地图内容，而非视觉通道本身。这种性能提升是以牺牲非对齐案例上的准确率为代价的。校准分析与指代链追踪进一步表明，模型依赖地图上的静态指代线索，而非追踪对话中共同基础如何通过交互逐步建立。这些模式在Qwen3-VL-8B-Instruct中最为明显，并在来自两个架构家族的其他四个模型中以不同程度显现。在表现出该偏差的模型中，无论是视觉还是文本形式呈现的地图内容，都被视为相互理解的证据，从而混淆了潜在共同基础与已建立的共同基础。

English

In collaborative dialogue, shared perception does not guarantee shared interpretation. Mutual understanding must be established through interaction. We investigate whether vision-language models (VLMs) can distinguish what could be shared from what has been shared between dialogue participants through grounding. We formulate this as an interpretation-matching task on 13,077 annotated reference expressions from HCRC MapTask dialogues, and evaluate VLMs under systematically controlled manipulations of dialogue context and map-information access. Our results show that providing authentic map images improves overall performance but shifts models toward over-predicting alignment. Textual descriptions of the same map content reproduce this bias, while non-informative images suppress alignment predictions entirely, indicating that the bias is driven by task-relevant map content, not the visual channel. This improvement comes at the cost of degraded accuracy on non-aligned cases. Calibration analysis and reference-chain tracking further suggest that models rely on static referential cues on the maps rather than tracking how grounding unfolds through dialogue history. We observe these patterns most clearly in Qwen3-VL-8B-Instruct and, to varying degrees, in four additional models from two architecture families. In models that exhibit the bias, map content, whether presented visually or textually, is treated as evidence of mutual understanding, conflating potential with established common ground.