量化与扩展后期交互检索模型的理论容量

摘要

使用MaxSim相似度函数的晚期交互检索模型在实证中表现出色，通常优于单向量稠密和稀疏检索模型。尽管已有实证发现，但关于MaxSim的理论表征能力及其与其他检索方法的比较仍知之甚少。本文通过构造证明，MaxSim相似度能够精确复制任意两个非负k稀疏向量（维度可能无限）的内积，仅需O(k)的表示空间。此外，存在某些相似度关系，MaxSim能够表达，而相同表示空间下的标准向量内积却无法做到。基于我们的理论框架，我们引入了Signed MaxSim，使晚期交互模型能够精确复制任意实值内积，而标准MaxSim被证明无法实现这一点。我们还表明，MaxSim可以充当软或（soft-OR）操作的聚合器，以及正合取范式（Conjunctive Normal Form）中逻辑表达式的评估器。我们的研究结果表明，对于任何非负向量，MaxSim至少与标准向量内积能力相当；而我们的扩展Signed MaxSim对于任意向量都具有同等能力。这两种相似度都具备内积无法复制的额外能力，这为晚期交互方法提供了最早的理论依据和量化分析之一。我们的理论发现得到了实证支持：在一个包含否定查询的检索任务中，Signed MaxSim在词汇偏移下将nDCG@10从0.597提升至1.000，在仅含否定词的查询上从0.008提升至0.788，显著优于标准ColBERT/MaxSim基线的域外性能。

English

Late-interaction retrieval models that use the MaxSim similarity function have shown strong empirical performance, often outperforming single-vector dense and sparse retrieval models. Despite these empirical findings, little is known about the theoretical representation power of MaxSim and how it compares to other retrieval approaches. This paper shows by construction that MaxSim similarity can exactly replicate the inner product between any two non-negative k-sparse vectors with possibly infinite dimension, requiring only O(k) representation space. Moreover, there exist similarities that MaxSim can express while standard vector inner products with the same representation space cannot. Leveraging our theoretical framework, we introduce Signed MaxSim which allows late-interaction models to exactly replicate any real-valued inner product, something we prove standard MaxSim is not capable of. We also show that MaxSim can act as an aggregation of soft-OR operations and as an evaluator of logical expressions in positive Conjunctive Normal Form. Our findings show that MaxSim is at least as capable as standard vector inner products for any non-negative vectors and our extension, Signed MaxSim, is as capable for any vectors. Both similarities possess additional capabilities that inner product cannot replicate, marking one of the first theoretical justifications and quantifications of late-interaction methods. Our theoretical findings are supported empirically: on a retrieval task featuring queries with negations, Signed MaxSim improves out-of-domain performance significantly over a standard ColBERT/MaxSim baseline with nDCG@10 increasing from 0.597 to 1.000 under a vocabulary shift and from 0.008 to 0.788 on negation-only queries.