MUSCLE：一種適用於相容LLM進化的模型更新策略

MUSCLE: A Model Update Strategy for Compatible LLM Evolution

July 12, 2024

作者: Jessica Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting-Yao Hu, Chun-Liang Li, Oncel Tuzel, Hadi Pouransari

cs.AI

摘要

大型語言模型（LLMs）經常因數據或架構變更而進行更新以提高性能。在更新模型時，開發人員通常專注於提高整體性能指標，對與先前模型版本兼容性的重視較少。然而，用戶通常會建立對特定機器學習模型的功能和能力的心智模型。他們必須隨著每次更新調整心智模型，這是一項繁瑣的任務，可能導致用戶不滿。在實踐中，微調的下游任務適配器依賴於預訓練的LLM基礎模型。當這些基礎模型進行更新時，這些面向用戶的下游任務模型可能出現實例回歸或負翻轉，即先前正確的實例現在被預測為錯誤。即使下游任務的訓練程序保持不變，這種情況仍會發生。我們的工作旨在以兩種方式為用戶提供無縫模型更新。首先，我們為與先前模型版本相容性概念提供評估指標，專門針對生成任務，但也適用於判別任務。我們觀察到在各種任務和模型更新上不同模型版本之間的回歸和不一致性。其次，我們提出了一種培訓策略，以最小化模型更新中不一致性的數量，包括訓練一個能夠增強任務微調語言模型的兼容性模型。我們成功將負翻轉（即先前模型版本正確但新模型不正確的實例）從Llama 1降低了高達40％至Llama 2。

English

Large Language Models (LLMs) are frequently updated due to data or architecture changes to improve their performance. When updating models, developers often focus on increasing overall performance metrics with less emphasis on being compatible with previous model versions. However, users often build a mental model of the functionality and capabilities of a particular machine learning model they are interacting with. They have to adapt their mental model with every update -- a draining task that can lead to user dissatisfaction. In practice, fine-tuned downstream task adapters rely on pretrained LLM base models. When these base models are updated, these user-facing downstream task models experience instance regression or negative flips -- previously correct instances are now predicted incorrectly. This happens even when the downstream task training procedures remain identical. Our work aims to provide seamless model updates to a user in two ways. First, we provide evaluation metrics for a notion of compatibility to prior model versions, specifically for generative tasks but also applicable for discriminative tasks. We observe regression and inconsistencies between different model versions on a diverse set of tasks and model updates. Second, we propose a training strategy to minimize the number of inconsistencies in model updates, involving training of a compatibility model that can enhance task fine-tuned language models. We reduce negative flips -- instances where a prior model version was correct, but a new model incorrect -- by up to 40% from Llama 1 to Llama 2.