Xetrieval: 從機制上解釋密集檢索

摘要

解釋為何密集檢索器會賦予高相關性分數仍然具有挑戰性，因為檢索決策是透過不透明的高維嵌入來進行的。現有的解釋通常關注表面信號，例如詞彙匹配、詞元對齊或事後文本理由，因此對於塑造密集檢索行為的嵌入層級潛在因素提供的洞察有限。我們提出 Xetrieval，這是一個用於解釋密集檢索的嵌入層級機制框架。Xetrieval 首先引入一個輕量級推理內化器，該內化器在嵌入空間中透過單次前向傳遞近似思維鏈推理，以推理導向資訊豐富句子嵌入，同時避免昂貴的自迴歸生成。然後，它將這些經推理增強的嵌入分解為稀疏、人類可解釋的特徵，每個特徵都與連貫的自然語言描述相關聯。透過匯總多個文件端視角的稀疏特徵重疊，Xetrieval 提供個別檢索決策的特徵層級解釋。在各種檢索器和基準測試上的實驗顯示，Xetrieval 能發現連貫的可解釋特徵，產生更強的配對層級干預效果，並支援任務層級的特徵引導。專案頁面和原始碼可在 https://hihiczx.github.io/Xetrieval 取得。

English

Explaining why dense retrievers assign high relevance scores remains challenging because retrieval decisions are made through opaque high-dimensional embeddings. Existing explanations often focus on surface signals, such as lexical matches, token alignments, or post-hoc textual rationales, and thus provide limited insight into the latent factors that shape dense retrieval behavior at the embedding level. We propose Xetrieval, an embedding-level mechanistic framework for explaining dense retrieval. Xetrieval first introduces a lightweight reasoning internalizer that approximates Chain-of-Thought reasoning directly in the embedding space with a single forward pass, enriching sentence embeddings with reasoning-oriented information while avoiding expensive autoregressive generation. It then decomposes these reasoning-enhanced embeddings into sparse, human-interpretable features, each associated with a coherent natural language description. By aggregating sparse feature overlaps across multiple document-side views, Xetrieval provides feature-level explanations of individual retrieval decisions. Experiments on diverse retrievers and benchmarks show that Xetrieval uncovers coherent interpretable features, yields stronger pair-level intervention effects, and supports task-level feature steering. The project page and source code are available at https://hihiczx.github.io/Xetrieval .