模仿專有LLM的虛假承諾

The False Promise of Imitating Proprietary LLMs

May 25, 2023

作者: Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, Dawn Song

cs.AI

摘要

一種新興的方法，可以廉價地改進較弱的語言模型，就是對其進行微調，使用來自較強模型的輸出，例如像ChatGPT這樣的專有系統（例如Alpaca、Self-Instruct等）。這種方法旨在廉價地模仿專有模型的能力，利用較弱的開源模型。在這項工作中，我們對這種方法進行了批判性分析。我們首先對一系列模仿ChatGPT的LM進行微調，使用不同的基礎模型大小（1.5B至13B）、數據來源和模仿數據量（0.3M至150M標記）。然後，我們使用眾包評估者和經典NLP基準來評估這些模型。最初，我們對我們的模仿模型的輸出質量感到驚訝 - 它們在遵循指示方面表現得更好，眾包工作者將其輸出評為與ChatGPT具有競爭力。然而，當進行更有針對性的自動評估時，我們發現，在模仿數據中未受到重大支持的任務上，模仿模型與基礎LM到ChatGPT之間的差距幾乎沒有縮小。我們展示了這些性能差異可能會逃過人類評估者的注意，因為模仿模型擅長模仿ChatGPT的風格，但不擅長模仿其事實性。總的來說，我們得出結論，模型模仿是一個虛假的承諾：在開放和封閉LM之間存在著實質性的能力差距，目前的方法只能通過大量的模仿數據或使用更有能力的基礎LM來彌合這一差距。因此，我們認為，改進開源模型的最有效行動是應對開發更好的基礎LM這一困難挑戰，而不是走捷徑，模仿專有系統。

English

An emerging method to cheaply improve a weaker language model is to finetune it on outputs from a stronger model, such as a proprietary system like ChatGPT (e.g., Alpaca, Self-Instruct, and others). This approach looks to cheaply imitate the proprietary model's capabilities using a weaker open-source model. In this work, we critically analyze this approach. We first finetune a series of LMs that imitate ChatGPT using varying base model sizes (1.5B--13B), data sources, and imitation data amounts (0.3M--150M tokens). We then evaluate the models using crowd raters and canonical NLP benchmarks. Initially, we were surprised by the output quality of our imitation models -- they appear far better at following instructions, and crowd workers rate their outputs as competitive with ChatGPT. However, when conducting more targeted automatic evaluations, we find that imitation models close little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. We show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, we conclude that model imitation is a false promise: there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. In turn, we argue that the highest leverage action for improving open-source models is to tackle the difficult challenge of developing better base LMs, rather than taking the shortcut of imitating proprietary systems.