MUSCLE：一种用于兼容LLM演化的模型更新策略

MUSCLE: A Model Update Strategy for Compatible LLM Evolution

July 12, 2024

作者: Jessica Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting-Yao Hu, Chun-Liang Li, Oncel Tuzel, Hadi Pouransari

cs.AI

摘要

大型语言模型（LLMs）经常因数据或架构变化而进行更新，以提高性能。在更新模型时，开发人员通常专注于提高整体性能指标，对与之前模型版本兼容性的重视较少。然而，用户经常会建立与特定机器学习模型的功能和能力相关的心理模型。他们必须随着每次更新调整自己的心理模型，这是一项耗费精力的任务，可能导致用户不满。在实践中，微调的下游任务适配器依赖于预训练的LLM基础模型。当这些基础模型更新时，这些面向用户的下游任务模型会出现实例回归或负翻转 -- 先前正确的实例现在被错误预测。即使下游任务的训练过程保持不变，这种情况仍会发生。我们的工作旨在以两种方式为用户提供无缝的模型更新。首先，我们为先前模型版本的兼容性概念提供评估指标，特别适用于生成任务，也适用于判别任务。我们观察到在各种任务和模型更新中不同模型版本之间的回归和不一致性。其次，我们提出了一种训练策略，以最小化模型更新中不一致性的数量，包括训练一个可以增强任务微调语言模型的兼容性模型。我们将负翻转 -- 先前模型版本正确的实例，但新模型错误的实例 -- 从Llama 1降低了高达40% 到Llama 2。

English

Large Language Models (LLMs) are frequently updated due to data or architecture changes to improve their performance. When updating models, developers often focus on increasing overall performance metrics with less emphasis on being compatible with previous model versions. However, users often build a mental model of the functionality and capabilities of a particular machine learning model they are interacting with. They have to adapt their mental model with every update -- a draining task that can lead to user dissatisfaction. In practice, fine-tuned downstream task adapters rely on pretrained LLM base models. When these base models are updated, these user-facing downstream task models experience instance regression or negative flips -- previously correct instances are now predicted incorrectly. This happens even when the downstream task training procedures remain identical. Our work aims to provide seamless model updates to a user in two ways. First, we provide evaluation metrics for a notion of compatibility to prior model versions, specifically for generative tasks but also applicable for discriminative tasks. We observe regression and inconsistencies between different model versions on a diverse set of tasks and model updates. Second, we propose a training strategy to minimize the number of inconsistencies in model updates, involving training of a compatibility model that can enhance task fine-tuned language models. We reduce negative flips -- instances where a prior model version was correct, but a new model incorrect -- by up to 40% from Llama 1 to Llama 2.