Rank1：資訊檢索中重排序的測試時計算

Rank1: Test-Time Compute for Reranking in Information Retrieval

February 25, 2025

作者: Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawrie, Benjamin Van Durme

cs.AI

摘要

我們推出了Rank1，這是首個利用測試時計算能力進行訓練的重排序模型。Rank1展示了在檢索中應用推理語言模型（如OpenAI的o1、Deepseek的R1等）進行蒸餾，以快速提升較小模型性能的可行性。我們收集並開源了一個包含超過60萬個來自MS MARCO查詢與段落R1推理軌跡的數據集。基於此數據集訓練的模型展現出：（1）在先進推理與指令遵循數據集上的頂尖性能；（2）由於能夠響應用戶輸入提示，在分佈外表現尤為出色；（3）擁有可解釋的推理鏈，可直接提供給用戶或基於RAG的系統。此外，我們還證明了這些模型的量化版本在減少計算/記憶體使用的情況下仍保持強勁性能。總體而言，Rank1證明了測試時計算能力為搜索提供了一種全新類型、可解釋且高效的重排序模型。

English

We introduce Rank1, the first reranking model trained to take advantage of test-time compute. Rank1 demonstrates the applicability within retrieval of using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for distillation in order to rapidly improve the performance of a smaller model. We gather and open-source a dataset of more than 600,000 examples of R1 reasoning traces from queries and passages in MS MARCO. Models trained on this dataset show: (1) state-of-the-art performance on advanced reasoning and instruction following datasets; (2) work remarkably well out of distribution due to the ability to respond to user-input prompts; and (3) have explainable reasoning chains that can be given to users or RAG-based systems. Further, we demonstrate that quantized versions of these models retain strong performance while using less compute/memory. Overall, Rank1 shows that test-time compute allows for a fundamentally new type of explainable and performant reranker model for search.