RecycleGPT：一种具有可回收模块的自回归语言模型

摘要

现有的大型语言模型必须运行K次才能生成K个标记的序列。在本文中，我们提出了RecycleGPT，这是一种具有快速解码速度的生成式语言模型，通过回收预先生成的模型状态，而无需在多个步骤中运行整个模型。我们的方法依赖于这样一个观察结果，即序列中相邻的标记通常具有很强的相关性，并且可以根据前面的标记合理猜测或推断出序列中的下一个标记。通过理论评估和对下游文本生成任务的实际测试，我们展示了我们的方法在降低推理延迟方面的有效性，实现了高达1.4倍的加速，同时保持高性能。

English

Existing large language models have to run K times to generate a sequence of K tokens. In this paper, we present RecycleGPT, a generative language model with fast decoding speed by recycling pre-generated model states without running the whole model in multiple steps. Our approach relies on the observation that adjacent tokens in a sequence usually have strong correlations and the next token in a sequence can be reasonably guessed or inferred based on the preceding ones. Through theoretical evaluations and practical tests on downstream text generation tasks, we demonstrate the effectiveness of our approach in lowering inference latency, achieving up to 1.4x speedup while preserving high performance.

RecycleGPT：一种具有可回收模块的自回归语言模型

RecycleGPT: An Autoregressive Language Model with Recyclable Module

摘要

Support