从失败中学习：通过难负样本的检索中心思维链实现统一多模态检索

摘要

统一多模态检索旨在识别满足由异构输入所表达的复杂用户意图的候选。尽管基于大型视觉-语言模型（LVLM）的检索器高效且可扩展，但直接编码原始多模态输入往往忽略细粒度的判别性线索，导致语义相似的候选之间产生混淆。近期方法通过生成思维链（CoT）推理来丰富查询表示，从而缓解这一局限。然而，此类推理通常仅源于查询本身：它解释了查询描述了什么，而非检索器误解了什么。我们认为，有效的检索推理应以检索反馈为条件。基于这一见解，我们提出了UniME-R1，一种嵌入器-顾问框架，该框架学习对初始检索到的候选进行推理，并生成以检索为中心的思维链（RC-CoT）。顾问逐一分析候选，以识别嵌入器混淆的判别性线索。如果目标出现在初始Top-k集合中，UniME-R1直接对候选重排序；否则，它生成RC-CoT以优化检索方向，并使用双模式嵌入器进行全语料库重新检索。为了训练该框架，我们挖掘困难负样本以模拟真实检索失败场景，联合优化直接检索和RC-CoT增强检索，并通过监督学习和面向检索的强化学习将顾问与检索结果对齐。在MMEB-V2和多样化的通用多模态检索基准上进行的大量实验表明，UniME-R1在强基线上持续提升了检索性能。

English

Unified multimodal retrieval aims to identify candidates that satisfy complex user intent expressed through heterogeneous inputs. Although Large Vision-Language Model (LVLM)-based retrievers are efficient and scalable, directly encoding raw multimodal inputs often misses fine-grained discriminative cues, leading to confusion among semantically similar candidates. Recent methods mitigate this limitation by generating Chain-of-Thought (CoT) rationales to enrich the query representation. However, such reasoning is typically derived from the query alone: it explains what the query describes, but not what the retriever misunderstands. We argue that effective retrieval reasoning should instead be conditioned on retrieval feedback. Based on this insight, we introduce UniME-R1, an embedder-adviser framework that learns to reason over initially retrieved candidates and generate Retrieval-Centric Chain-of-Thought (RC-CoT). The adviser analyzes candidates individually to identify the discriminative cues confused by the embedder. If the target appears in the initial top-k set, UniME-R1 directly reranks the candidates; otherwise, it generates RC-CoT to refine the retrieval direction and performs full-corpus re-retrieval with a dual-mode embedder. To train the framework, we mine hard negatives to simulate realistic retrieval failures, jointly optimize direct retrieval and RC-CoT-augmented retrieval, and align the adviser with retrieval outcomes through supervised learning and retrieval-oriented reinforcement learning. Extensive experiments on MMEB-V2 and a diverse set of general multimodal retrieval benchmarks demonstrate that UniME-R1 consistently improves retrieval performance over strong baselines.