從失敗中學習：基於困難負樣本的檢索中心思維鏈用於統一多模態檢索

摘要

統一多模態檢索旨在從異質輸入中識別能滿足複雜使用者意圖的候選項目。儘管基於大型視覺語言模型（LVLM）的檢索器具備高效能與可擴展性，直接編碼原始多模態輸入往往會遺漏細粒度的鑑別線索，導致語意相似的候選項目之間產生混淆。近期方法透過生成思維鏈（CoT）理由來豐富查詢表示，以緩解此限制。然而，此類推理通常僅源於查詢本身：它解釋了查詢所描述的內容，卻未解釋檢索器誤解了什麼。我們主張，有效的檢索推理應改以檢索回饋為條件。基於此觀點，我們提出 UniME-R1，這是一個嵌入器-顧問框架，學習對初始檢索到的候選項目進行推理，並生成以檢索為中心的思維鏈（RC-CoT）。顧問逐一分析候選項目，以辨識被嵌入器混淆的鑑別線索。若目標出現在初始的前 k 個集合中，UniME-R1 直接對候選項目重新排序；否則，它生成 RC-CoT 以修正檢索方向，並使用雙模式嵌入器對整個語料庫重新檢索。為了訓練此框架，我們挖掘困難負樣本以模擬真實的檢索失敗情況，聯合優化直接檢索與 RC-CoT 增強的檢索，並透過監督式學習及以檢索為導向的強化學習，使顧問與檢索結果對齊。在 MMEB-V2 及多樣化的通用多模態檢索基準上的大量實驗顯示，UniME-R1 在強基線方法之上持續提升了檢索效能。

English

Unified multimodal retrieval aims to identify candidates that satisfy complex user intent expressed through heterogeneous inputs. Although Large Vision-Language Model (LVLM)-based retrievers are efficient and scalable, directly encoding raw multimodal inputs often misses fine-grained discriminative cues, leading to confusion among semantically similar candidates. Recent methods mitigate this limitation by generating Chain-of-Thought (CoT) rationales to enrich the query representation. However, such reasoning is typically derived from the query alone: it explains what the query describes, but not what the retriever misunderstands. We argue that effective retrieval reasoning should instead be conditioned on retrieval feedback. Based on this insight, we introduce UniME-R1, an embedder-adviser framework that learns to reason over initially retrieved candidates and generate Retrieval-Centric Chain-of-Thought (RC-CoT). The adviser analyzes candidates individually to identify the discriminative cues confused by the embedder. If the target appears in the initial top-k set, UniME-R1 directly reranks the candidates; otherwise, it generates RC-CoT to refine the retrieval direction and performs full-corpus re-retrieval with a dual-mode embedder. To train the framework, we mine hard negatives to simulate realistic retrieval failures, jointly optimize direct retrieval and RC-CoT-augmented retrieval, and align the adviser with retrieval outcomes through supervised learning and retrieval-oriented reinforcement learning. Extensive experiments on MMEB-V2 and a diverse set of general multimodal retrieval benchmarks demonstrate that UniME-R1 consistently improves retrieval performance over strong baselines.