学习为大型语言模型检索上下文示例

Learning to Retrieve In-Context Examples for Large Language Models

July 14, 2023

作者: Liang Wang, Nan Yang, Furu Wei

cs.AI

摘要

大型语言模型（LLMs）展示了它们能够学习上下文，使其能够根据少量输入-输出示例执行各种任务的能力。然而，上下文学习的有效性在很大程度上取决于所选示例的质量。在本文中，我们提出了一个新颖的框架，用于迭代训练密集检索器，该检索器能够识别LLMs的高质量上下文示例。我们的框架最初训练一个基于LLM反馈的奖励模型来评估候选示例的质量，然后进行知识蒸馏以训练基于双编码器的密集检索器。我们在30个任务套件上的实验表明，我们的框架显著提高了上下文学习性能。此外，我们展示了我们的框架对训练期间未见任务的泛化能力。深入分析显示，我们的模型通过检索具有相似模式的示例来提高性能，并且这种增益在不同大小的LLMs之间保持一致。

English

Large language models (LLMs) have demonstrated their ability to learn in-context, allowing them to perform various tasks based on a few input-output examples. However, the effectiveness of in-context learning is heavily reliant on the quality of the selected examples. In this paper, we propose a novel framework to iteratively train dense retrievers that can identify high-quality in-context examples for LLMs. Our framework initially trains a reward model based on LLM feedback to evaluate the quality of candidate examples, followed by knowledge distillation to train a bi-encoder based dense retriever. Our experiments on a suite of 30 tasks demonstrate that our framework significantly enhances in-context learning performance. Furthermore, we show the generalization ability of our framework to unseen tasks during training. An in-depth analysis reveals that our model improves performance by retrieving examples with similar patterns, and the gains are consistent across LLMs of varying sizes.