眼見不等於共享：部分視覺語言模型在非對稱對話中高估共同背景

摘要

在协作对话中，共享感知并不等同于共享解读。共同理解必须通过交互建立。我们探究视觉-语言模型（VLM）是否能通过共同基础（grounding）区分对话参与者之间"可能共享"与"已经共享"的信息。为此，我们利用 HCRC MapTask 对话语料库中 13,077 个带标注的指称表达式，构建了一个"解释匹配任务"，并在对话上下文与地图信息访问的系统性受控操纵条件下评估 VLM。结果表明，提供真实地图图像虽能提升整体性能，但会使模型过度预测对齐。相同的文本化地图内容复现了这一偏差，而无信息量的图像则完全抑制了对齐预测，说明该偏差源于任务相关的地图内容，而非视觉通道本身。这种提升以牺牲非对齐案例的准确率为代价。校准分析与指称链追踪进一步表明，模型依赖地图上静态的指称线索，而非通过对话历史追踪共同基础的建立过程。这些现象在 Qwen3-VL-8B-Instruct 中最为显著，并在两个架构家族的额外四个模型中以不同程度呈现。在表现出这一偏差的模型中，地图内容（无论是视觉还是文本呈现）均被视为共同理解的证据，从而混淆了潜在共同基础与已建立共同基础。

English

In collaborative dialogue, shared perception does not guarantee shared interpretation. Mutual understanding must be established through interaction. We investigate whether vision-language models (VLMs) can distinguish what could be shared from what has been shared between dialogue participants through grounding. We formulate this as an interpretation-matching task on 13,077 annotated reference expressions from HCRC MapTask dialogues, and evaluate VLMs under systematically controlled manipulations of dialogue context and map-information access. Our results show that providing authentic map images improves overall performance but shifts models toward over-predicting alignment. Textual descriptions of the same map content reproduce this bias, while non-informative images suppress alignment predictions entirely, indicating that the bias is driven by task-relevant map content, not the visual channel. This improvement comes at the cost of degraded accuracy on non-aligned cases. Calibration analysis and reference-chain tracking further suggest that models rely on static referential cues on the maps rather than tracking how grounding unfolds through dialogue history. We observe these patterns most clearly in Qwen3-VL-8B-Instruct and, to varying degrees, in four additional models from two architecture families. In models that exhibit the bias, map content, whether presented visually or textually, is treated as evidence of mutual understanding, conflating potential with established common ground.