檢索增強生成的上下文調整

摘要

大型語言模型（LLMs）具有卓越的能力，僅需少量示例即可解決新任務，但它們需要訪問正確的工具。檢索增強生成（RAG）通過檢索給定任務的相關工具列表來解決這個問題。然而，RAG的工具檢索步驟要求查詢中包含所有必要的信息。這是一個限制，因為語義搜索，廣泛採用的工具檢索方法，在查詢不完整或缺乏上下文時可能失敗。為了解決這個限制，我們提出了RAG的上下文調整，它採用智能上下文檢索系統來提取改進工具檢索和計劃生成的相關信息。我們的輕量級上下文檢索模型使用數值、分類和習慣使用信號來檢索和排名上下文項目。我們的實證結果表明，上下文調整顯著增強了語義搜索，在上下文檢索和工具檢索任務的Recall@K分別實現了3.5倍和1.5倍的改進，並導致基於LLM的計劃器準確性提高了11.6%。此外，我們展示了我們提出的輕量級模型使用Reciprocal Rank Fusion（RRF）與LambdaMART優於基於GPT-4的檢索。此外，我們觀察到在工具檢索後，計劃生成時的上下文增強有助於減少幻覺。

English

Large language models (LLMs) have the remarkable ability to solve new tasks with just a few examples, but they need access to the right tools. Retrieval Augmented Generation (RAG) addresses this problem by retrieving a list of relevant tools for a given task. However, RAG's tool retrieval step requires all the required information to be explicitly present in the query. This is a limitation, as semantic search, the widely adopted tool retrieval method, can fail when the query is incomplete or lacks context. To address this limitation, we propose Context Tuning for RAG, which employs a smart context retrieval system to fetch relevant information that improves both tool retrieval and plan generation. Our lightweight context retrieval model uses numerical, categorical, and habitual usage signals to retrieve and rank context items. Our empirical results demonstrate that context tuning significantly enhances semantic search, achieving a 3.5-fold and 1.5-fold improvement in Recall@K for context retrieval and tool retrieval tasks respectively, and resulting in an 11.6% increase in LLM-based planner accuracy. Additionally, we show that our proposed lightweight model using Reciprocal Rank Fusion (RRF) with LambdaMART outperforms GPT-4 based retrieval. Moreover, we observe context augmentation at plan generation, even after tool retrieval, reduces hallucination.

檢索增強生成的上下文調整

Context Tuning for Retrieval Augmented Generation

摘要

Support