BiCA：基于引文感知困难负样本的高效生物医学密集检索

摘要

硬负样本对于训练高效检索模型至关重要。硬负样本挖掘通常依赖于使用基于余弦距离等相似性度量的交叉编码器或静态嵌入模型对文档进行排序。在生物医学和科学领域，由于难以区分源文档与硬负样本文档，硬负样本挖掘变得尤为困难。然而，被引文献天然与源文档具有上下文关联性却非重复内容，这使其成为理想的硬负样本。本研究提出BiCA：基于引文感知硬负样本的生物医学稠密检索方法，通过利用20,000篇PubMed文献中的引文链接进行硬负样本挖掘，以改进领域专用的小型稠密检索器。我们使用这些引文指导的负样本对GTE_small和GTE_Base模型进行微调，在BEIR基准的域内和域外任务中通过nDCG@10指标观察到零样本稠密检索性能的持续提升，并在LoTTE的长尾主题任务中通过Success@5指标超越基线。我们的研究结果揭示了利用文档链接结构生成高信息量负样本的潜力，仅需少量微调即可实现最先进的性能，为高数据效率的领域适应提供了可行路径。

English

Hard negatives are essential for training effective retrieval models. Hard-negative mining typically relies on ranking documents using cross-encoders or static embedding models based on similarity metrics such as cosine distance. Hard negative mining becomes challenging for biomedical and scientific domains due to the difficulty in distinguishing between source and hard negative documents. However, referenced documents naturally share contextual relevance with the source document but are not duplicates, making them well-suited as hard negatives. In this work, we propose BiCA: Biomedical Dense Retrieval with Citation-Aware Hard Negatives, an approach for hard-negative mining by utilizing citation links in 20,000 PubMed articles for improving a domain-specific small dense retriever. We fine-tune the GTE_small and GTE_Base models using these citation-informed negatives and observe consistent improvements in zero-shot dense retrieval using nDCG@10 for both in-domain and out-of-domain tasks on BEIR and outperform baselines on long-tailed topics in LoTTE using Success@5. Our findings highlight the potential of leveraging document link structure to generate highly informative negatives, enabling state-of-the-art performance with minimal fine-tuning and demonstrating a path towards highly data-efficient domain adaptation.

BiCA：基于引文感知困难负样本的高效生物医学密集检索

BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives

摘要

Support