Jina-ColBERT-v2：通用多语言后交互检索器

Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retriever

August 29, 2024

作者: Rohan Jha, Bo Wang, Michael Günther, Saba Sturua, Mohammad Kalim Akram, Han Xiao

cs.AI

摘要

多向量密集模型，例如ColBERT，在信息检索中证明了其高效性。ColBERT的后期交互评分近似于交叉编码器中所见的联合查询-文档注意力，同时保持推理效率接近传统的密集检索模型，这要归功于其双编码器架构以及最近在索引和搜索方面的优化。在本文中，我们对ColBERT模型架构和训练流程进行了几项改进，利用了在更成熟的单向量嵌入模型范式中取得成功的技术，特别是适用于异构多语言数据的技术。我们的新模型Jina-ColBERT-v2在各种英语和多语言检索任务中展现出强大的性能，同时与先前模型相比，还将存储需求减少了高达50%。

English

Multi-vector dense models, such as ColBERT, have proven highly effective in information retrieval. ColBERT's late interaction scoring approximates the joint query-document attention seen in cross-encoders while maintaining inference efficiency closer to traditional dense retrieval models, thanks to its bi-encoder architecture and recent optimizations in indexing and search. In this paper, we introduce several improvements to the ColBERT model architecture and training pipeline, leveraging techniques successful in the more established single-vector embedding model paradigm, particularly those suited for heterogeneous multilingual data. Our new model, Jina-ColBERT-v2, demonstrates strong performance across a range of English and multilingual retrieval tasks, while also cutting storage requirements by up to 50% compared to previous models.