MULTI3IR：一个面向多视角、多领域、多模态信息检索的基准测试

摘要

信息检索（IR）日益面向允许多元视角的开放式查询。然而，现有的 IR 基准主要聚焦于封闭式查询，即便是开放式基准，其查询所对应的支撑文档大多也局限于单一学科领域和单一模态。我们提出了 Multi³IR，这是一个基准测试，用于评估检索器如何覆盖跨领域、跨模态的开放式查询的多层面视角。该基准包含 104.9 万个 Stack Exchange 查询，每个查询都标注了用于刻画其隐含视角的视角描述。我们进一步提出了 SPIN，一种参数高效且标签高效的方法，通过学习噪声向量将嵌入引导至多样而有意义的语义方向。实验表明，现有的多模态检索器存在单视角偏差，而 SPIN 在 Multi³IR 上显著提升了视角覆盖率，并且能够很好地泛化到未见过的开放式 IR 基准。数据集和实验代码可在 https://github.com/seokwon99/Multi3IR 获取。

English

Information retrieval (IR) increasingly targets open-ended queries that admit diverse perspectives. Existing IR benchmarks, however, focus primarily on closed-ended queries, while even open-ended benchmarks largely consist of queries whose supporting documents span a single subject domain and modality. We introduce Multi^3IR, a benchmark that evaluates how well retrievers cover the multifaceted perspectives of open-ended queries across diverse domains and modalities. It comprises 104.9K Stack Exchange queries, each annotated with perspective descriptions that capture the query's implicit viewpoints. We further propose SPIN, a parameter- and label-efficient method that learns noise vectors to steer embeddings toward diverse yet meaningful semantic directions. Experiments show that existing multimodal retrievers suffer from single-perspective bias, while SPIN substantially improves perspective coverage on Multi^3IR and generalizes well to unseen open-ended IR benchmarks. The dataset and experimental code are available at https://github.com/seokwon99/Multi3IR.