BiCA：基于引文感知难负例的高效生物医学稠密检索

摘要

硬負樣本對於訓練有效的檢索模型至關重要。硬負樣本挖掘通常依賴於使用基於餘弦距離等相似性度量的交叉編碼器或靜態嵌入模型對文檔進行排序。在生物醫學和科學領域，由於難以區分源文檔與硬負樣本文檔，硬負樣本挖掘變得具有挑戰性。然而，被引文獻天然與源文檔具有上下文相關性卻非重複內容，這使其成為理想的硬負樣本。本研究提出BiCA：具備引文感知硬負樣本的生物醫學密集檢索方法，通過利用20,000篇PubMed文獻中的引文鏈接進行硬負樣本挖掘，以改進領域專用的小型密集檢索器。我們使用這些引文指導的負樣本對GTE_small和GTE_Base模型進行微調，在BEIR數據集的域內和域外任務中通過nDCG@10指標觀察到零樣本密集檢索的持續提升，並在LoTTE數據集的長尾主題上通過Success@5指標超越基線。我們的研究結果凸顯了利用文檔鏈接結構生成高信息量負樣本的潛力，僅需極少微調即可實現最先進的性能，為實現高數據效率的領域自適應開闢了新途徑。

English

Hard negatives are essential for training effective retrieval models. Hard-negative mining typically relies on ranking documents using cross-encoders or static embedding models based on similarity metrics such as cosine distance. Hard negative mining becomes challenging for biomedical and scientific domains due to the difficulty in distinguishing between source and hard negative documents. However, referenced documents naturally share contextual relevance with the source document but are not duplicates, making them well-suited as hard negatives. In this work, we propose BiCA: Biomedical Dense Retrieval with Citation-Aware Hard Negatives, an approach for hard-negative mining by utilizing citation links in 20,000 PubMed articles for improving a domain-specific small dense retriever. We fine-tune the GTE_small and GTE_Base models using these citation-informed negatives and observe consistent improvements in zero-shot dense retrieval using nDCG@10 for both in-domain and out-of-domain tasks on BEIR and outperform baselines on long-tailed topics in LoTTE using Success@5. Our findings highlight the potential of leveraging document link structure to generate highly informative negatives, enabling state-of-the-art performance with minimal fine-tuning and demonstrating a path towards highly data-efficient domain adaptation.

BiCA：基于引文感知难负例的高效生物医学稠密检索

BiCA: Effective Biomedical Dense Retrieval with Citation-Aware Hard Negatives

摘要

Support