密集檢索器在簡單查詢上可能失效：揭示嵌入的粒度困境

摘要

本研究聚焦于文本编码器的一个已观察到的局限性：嵌入可能无法识别语义中的细粒度实体或事件，导致即使在简单案例中也无法实现密集检索。为了探究此类行为，我们首先引入了一个新的中文评估数据集，名为CapRetrieval，其段落为图像描述，查询则是以多种形式询问实体或事件的短语。零样本评估表明，无论训练来源或模型规模如何，编码器在这些细粒度匹配上均可能失败。为了寻求改进，我们继而采用我们提出的数据生成策略对编码器进行微调，从而在CapRetrieval上取得了最佳性能。在此过程中，我们进一步识别出一个粒度困境问题，即嵌入在表达细粒度显著性的同时与整体语义对齐所面临的挑战。本工作中的数据集、代码及模型已公开发布于https://github.com/lxucs/CapRetrieval。

English

This work focuses on an observed limitation of text encoders: embeddings may not be able to recognize fine-grained entities or events within the semantics, resulting in failed dense retrieval on even simple cases. To examine such behaviors, we first introduce a new evaluation dataset in Chinese, named CapRetrieval, whose passages are image captions, and queries are phrases inquiring entities or events in various forms. Zero-shot evaluation suggests that encoders may fail on these fine-grained matching, regardless of training sources or model sizes. Aiming for enhancement, we proceed to finetune encoders with our proposed data generation strategies, which obtains the best performance on CapRetrieval. Within this process, we further identify an issue of granularity dilemma, a challenge for embeddings to express fine-grained salience while aligning with overall semantics. Our dataset, code and models in this work are publicly released at https://github.com/lxucs/CapRetrieval.

密集檢索器在簡單查詢上可能失效：揭示嵌入的粒度困境

Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddings

摘要

Support