密集检索器可能在简单查询上失效：揭示嵌入的粒度困境

摘要

本研究聚焦于文本编码器的一个显著局限：嵌入向量可能无法识别语义中的细粒度实体或事件，导致即使在简单案例中也难以实现有效的密集检索。为探究这一现象，我们首先引入了一个全新的中文评估数据集——CapRetrieval，其段落内容为图像描述，查询则采用多种形式询问实体或事件。零样本评估显示，无论训练来源或模型规模如何，编码器在这些细粒度匹配任务上均可能表现不佳。为寻求改进，我们继而采用提出的数据生成策略对编码器进行微调，从而在CapRetrieval上取得了最佳性能。在此过程中，我们进一步识别出“粒度困境”问题，即嵌入向量在表达细粒度显著性的同时，还需与整体语义保持一致，这一挑战尤为突出。本研究的全部数据集、代码及模型已公开发布于https://github.com/lxucs/CapRetrieval。

English

This work focuses on an observed limitation of text encoders: embeddings may not be able to recognize fine-grained entities or events within the semantics, resulting in failed dense retrieval on even simple cases. To examine such behaviors, we first introduce a new evaluation dataset in Chinese, named CapRetrieval, whose passages are image captions, and queries are phrases inquiring entities or events in various forms. Zero-shot evaluation suggests that encoders may fail on these fine-grained matching, regardless of training sources or model sizes. Aiming for enhancement, we proceed to finetune encoders with our proposed data generation strategies, which obtains the best performance on CapRetrieval. Within this process, we further identify an issue of granularity dilemma, a challenge for embeddings to express fine-grained salience while aligning with overall semantics. Our dataset, code and models in this work are publicly released at https://github.com/lxucs/CapRetrieval.

密集检索器可能在简单查询上失效：揭示嵌入的粒度困境

Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddings

摘要

Support