RecycleGPT：具有可回收模組的自回歸語言模型

摘要

現有的大型語言模型必須運行 K 次才能生成 K 個標記的序列。在本文中，我們提出了RecycleGPT，一種具有快速解碼速度的生成式語言模型，通過重複使用預先生成的模型狀態而無需在多個步驟中運行整個模型。我們的方法基於一個觀察，即序列中相鄰的標記通常具有很強的相關性，並且序列中的下一個標記可以根據前面的標記合理地猜測或推斷。通過理論評估和對下游文本生成任務的實際測試，我們展示了我們的方法在降低推理延遲方面的有效性，實現高達1.4倍的加速，同時保持高性能。

English

Existing large language models have to run K times to generate a sequence of K tokens. In this paper, we present RecycleGPT, a generative language model with fast decoding speed by recycling pre-generated model states without running the whole model in multiple steps. Our approach relies on the observation that adjacent tokens in a sequence usually have strong correlations and the next token in a sequence can be reasonably guessed or inferred based on the preceding ones. Through theoretical evaluations and practical tests on downstream text generation tasks, we demonstrate the effectiveness of our approach in lowering inference latency, achieving up to 1.4x speedup while preserving high performance.

RecycleGPT：具有可回收模組的自回歸語言模型

RecycleGPT: An Autoregressive Language Model with Recyclable Module

摘要

Support