检索增强生成的上下文调整

摘要

大型语言模型（LLMs）具有出色的能力，仅凭几个示例就能解决新任务，但它们需要访问正确的工具。检索增强生成（RAG）通过检索给定任务的相关工具列表来解决这一问题。然而，RAG的工具检索步骤要求查询中包含所有必要的信息。这是一个限制，因为语义搜索，被广泛采纳的工具检索方法，在查询不完整或缺乏上下文时可能失败。为了解决这一限制，我们提出了RAG的上下文调整，它采用智能上下文检索系统来获取改进工具检索和计划生成的相关信息。我们的轻量级上下文检索模型使用数字、分类和习惯使用信号来检索和排名上下文项。我们的实证结果表明，上下文调整显著增强了语义搜索，在上下文检索和工具检索任务的Recall@K分别实现了3.5倍和1.5倍的改进，并导致基于LLM的计划器准确度提高了11.6%。此外，我们展示了我们提出的轻量级模型使用Reciprocal Rank Fusion（RRF）与LambdaMART优于基于GPT-4的检索。此外，我们观察到在工具检索后，计划生成时的上下文增强减少了虚构现象。

English

Large language models (LLMs) have the remarkable ability to solve new tasks with just a few examples, but they need access to the right tools. Retrieval Augmented Generation (RAG) addresses this problem by retrieving a list of relevant tools for a given task. However, RAG's tool retrieval step requires all the required information to be explicitly present in the query. This is a limitation, as semantic search, the widely adopted tool retrieval method, can fail when the query is incomplete or lacks context. To address this limitation, we propose Context Tuning for RAG, which employs a smart context retrieval system to fetch relevant information that improves both tool retrieval and plan generation. Our lightweight context retrieval model uses numerical, categorical, and habitual usage signals to retrieve and rank context items. Our empirical results demonstrate that context tuning significantly enhances semantic search, achieving a 3.5-fold and 1.5-fold improvement in Recall@K for context retrieval and tool retrieval tasks respectively, and resulting in an 11.6% increase in LLM-based planner accuracy. Additionally, we show that our proposed lightweight model using Reciprocal Rank Fusion (RRF) with LambdaMART outperforms GPT-4 based retrieval. Moreover, we observe context augmentation at plan generation, even after tool retrieval, reduces hallucination.

检索增强生成的上下文调整

Context Tuning for Retrieval Augmented Generation

摘要

Support