模仿专有LLM的虚假承诺
The False Promise of Imitating Proprietary LLMs
May 25, 2023
作者: Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, Dawn Song
cs.AI
摘要
一种新兴的廉价改进较弱语言模型的方法是在更强大的模型(如ChatGPT这样的专有系统,例如Alpaca、Self-Instruct等)的输出上进行微调。这种方法旨在通过较弱的开源模型廉价模仿专有模型的能力。在这项工作中,我们对这种方法进行了批判性分析。我们首先微调了一系列模仿ChatGPT的LM,使用不同的基础模型大小(1.5B至13B)、数据来源和模仿数据量(0.3M至150M标记)。然后,我们使用众包评估员和经典NLP基准来评估这些模型。最初,我们对我们的模仿模型的输出质量感到惊讶——它们似乎更擅长遵循指令,并且众包工作者评价它们的输出与ChatGPT相媲美。然而,当进行更有针对性的自动评估时,我们发现在模仿数据中没有得到充分支持的任务上,模仿模型几乎没有缩小基础LM与ChatGPT之间的差距。我们表明,这些性能差异可能会逃过人类评估员的注意,因为模仿模型擅长模仿ChatGPT的风格,但不擅长模仿其事实性。总的来说,我们得出结论认为,模型模仿是一种虚假承诺:在开源和闭源LM之间存在着实质性的能力差距,目前的方法只能通过大量的模仿数据或使用更有能力的基础LM来弥合这一差距。因此,我们认为,改进开源模型的最有效举措是解决开发更好的基础LM这一困难挑战,而不是采取模仿专有系统的捷径。
English
An emerging method to cheaply improve a weaker language model is to finetune
it on outputs from a stronger model, such as a proprietary system like ChatGPT
(e.g., Alpaca, Self-Instruct, and others). This approach looks to cheaply
imitate the proprietary model's capabilities using a weaker open-source model.
In this work, we critically analyze this approach. We first finetune a series
of LMs that imitate ChatGPT using varying base model sizes (1.5B--13B), data
sources, and imitation data amounts (0.3M--150M tokens). We then evaluate the
models using crowd raters and canonical NLP benchmarks. Initially, we were
surprised by the output quality of our imitation models -- they appear far
better at following instructions, and crowd workers rate their outputs as
competitive with ChatGPT. However, when conducting more targeted automatic
evaluations, we find that imitation models close little to none of the gap from
the base LM to ChatGPT on tasks that are not heavily supported in the imitation
data. We show that these performance discrepancies may slip past human raters
because imitation models are adept at mimicking ChatGPT's style but not its
factuality. Overall, we conclude that model imitation is a false promise: there
exists a substantial capabilities gap between open and closed LMs that, with
current methods, can only be bridged using an unwieldy amount of imitation data
or by using more capable base LMs. In turn, we argue that the highest leverage
action for improving open-source models is to tackle the difficult challenge of
developing better base LMs, rather than taking the shortcut of imitating
proprietary systems.