量化與擴展晚期交互檢索模型的理論容量

摘要

使用 MaxSim 相似度函数的晚期交互检索模型在实证中展现出强大性能，通常优于单向量稠密检索和稀疏检索模型。尽管已有实证发现，但关于 MaxSim 的表示能力及其与其他检索方法的理论对比仍知之甚少。本文通过构造方法证明，MaxSim 相似度能够精确复现任意两个非负 k-稀疏向量（可能具有无限维度）之间的内积，且仅需 O(k) 表示空间。此外，存在某些相似度关系是 MaxSim 可以表达，而相同表示空间下的标准向量内积无法表达的。基于我们的理论框架，我们引入了 Signed MaxSim，使晚期交互模型能够精确复现任意实值内积，而标准 MaxSim 被证明无法实现这一点。我们还表明，MaxSim 可作为软 OR 操作的聚合函数，以及正合取范式逻辑表达式的评估器。研究结果表明，对于任何非负向量，MaxSim 至少与标准向量内积能力相当，而我们的扩展 Signed MaxSim 对于任何向量均具备同等能力。这两种相似度均具有内积无法复现的额外能力，这为晚期交互方法提供了首批理论依据与量化分析。理论发现得到实证支持：在包含否定查询的检索任务中，Signed MaxSim 相比标准 ColBERT/MaxSim 基线显著提升了域外性能，当词汇偏移时 nDCG@10 从 0.597 提升至 1.000，在仅含否定查询时从 0.008 提升至 0.788。

English

Late-interaction retrieval models that use the MaxSim similarity function have shown strong empirical performance, often outperforming single-vector dense and sparse retrieval models. Despite these empirical findings, little is known about the theoretical representation power of MaxSim and how it compares to other retrieval approaches. This paper shows by construction that MaxSim similarity can exactly replicate the inner product between any two non-negative k-sparse vectors with possibly infinite dimension, requiring only O(k) representation space. Moreover, there exist similarities that MaxSim can express while standard vector inner products with the same representation space cannot. Leveraging our theoretical framework, we introduce Signed MaxSim which allows late-interaction models to exactly replicate any real-valued inner product, something we prove standard MaxSim is not capable of. We also show that MaxSim can act as an aggregation of soft-OR operations and as an evaluator of logical expressions in positive Conjunctive Normal Form. Our findings show that MaxSim is at least as capable as standard vector inner products for any non-negative vectors and our extension, Signed MaxSim, is as capable for any vectors. Both similarities possess additional capabilities that inner product cannot replicate, marking one of the first theoretical justifications and quantifications of late-interaction methods. Our theoretical findings are supported empirically: on a retrieval task featuring queries with negations, Signed MaxSim improves out-of-domain performance significantly over a standard ColBERT/MaxSim baseline with nDCG@10 increasing from 0.597 to 1.000 under a vocabulary shift and from 0.008 to 0.788 on negation-only queries.