朝向建立聯邦式 GPT：聯邦式指導調校

Towards Building the Federated GPT: Federated Instruction Tuning

May 9, 2023

作者: Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Guoyin Wang, Yiran Chen

cs.AI

摘要

儘管「指導調整」的生成式大型語言模型（LLMs）展現出令人印象深刻的泛化能力，可以適應新任務，但訓練階段嚴重依賴大量多樣且高質量的指導數據（例如ChatGPT和GPT-4）。不幸的是，獲取高質量數據，特別是人類撰寫的數據，可能會在成本和可取得性方面帶來重大挑戰。此外，與隱私相關的擔憂可能進一步限制對這些數據的訪問，使獲取數據的過程變得複雜且微妙。因此，這阻礙了調整模型的泛化能力，並可能限制其在某些情境下的有效性。為了應對這個問題，我們的研究引入了一種名為聯邦指導調整（FedIT）的新方法，該方法利用聯邦學習（FL）作為調整LLMs的指導框架。這標誌著FL為LLMs的指導調整的首次探索。這一點尤為重要，因為文本數據主要由最終用戶生成。因此，必須設計和適應FL方法，以有效利用這些用戶在本地設備上存儲的多樣指導，同時保護隱私並確保數據安全。在本文中，通過進行廣泛使用的GPT-4自我評估，我們展示了通過利用客戶端端的異構和多樣指導集合，使用提出的框架FedIT，相較於僅具有有限本地指導的集中式訓練，我們提高了LLMs的性能。此外，在本文中，我們開發了一個名為Shepherd的Github存儲庫。該存儲庫提供了一個基礎框架，用於探索使用跨不同類別的異構指導進行LLMs的聯邦微調。

English

While ``instruction-tuned" generative large language models (LLMs) have demonstrated an impressive ability to generalize to new tasks, the training phases heavily rely on large amounts of diverse and high-quality instruction data (such as ChatGPT and GPT-4). Unfortunately, acquiring high-quality data, especially when it comes to human-written data, can pose significant challenges both in terms of cost and accessibility. Moreover, concerns related to privacy can further limit access to such data, making the process of obtaining it a complex and nuanced undertaking. Consequently, this hinders the generality of the tuned models and may restrict their effectiveness in certain contexts. To tackle this issue, our study introduces a new approach called Federated Instruction Tuning (FedIT), which leverages federated learning (FL) as the learning framework for the instruction tuning of LLMs. This marks the first exploration of FL-based instruction tuning for LLMs. This is especially important since text data is predominantly generated by end users. Therefore, it is imperative to design and adapt FL approaches to effectively leverage these users' diverse instructions stored on local devices, while preserving privacy and ensuring data security. In the current paper, by conducting widely used GPT-4 auto-evaluation, we demonstrate that by exploiting the heterogeneous and diverse sets of instructions on the client's end with the proposed framework FedIT, we improved the performance of LLMs compared to centralized training with only limited local instructions. Further, in this paper, we developed a Github repository named Shepherd. This repository offers a foundational framework for exploring federated fine-tuning of LLMs using heterogeneous instructions across diverse categories.