基於LLM自動化的聯邦學習網絡解決方案

A Web-Based Solution for Federated Learning with LLM-Based Automation

August 23, 2024

作者: Chamith Mawela, Chaouki Ben Issaid, Mehdi Bennis

cs.AI

摘要

聯邦學習（FL）為分佈式設備間協作機器學習提供了一種有前途的方法。然而，由於構建可靠的通信架構的複雜性以及對機器學習和網絡編程專業知識的需求，其應用受到了阻礙。本文提出了一個全面的解決方案，簡化了FL任務的協調，同時整合了基於意圖的自動化。我們開發了一個用戶友好的Web應用程序，支持聯邦平均（FedAvg）算法，使用戶可以通過直觀的界面配置參數。後端解決方案有效地管理參數服務器與邊緣節點之間的通信。我們還實現了模型壓縮和排程算法，以優化FL的性能。此外，我們利用在定制數據集上訓練的微調語言模型（LLM）探索了FL中的基於意圖的自動化，使用戶可以使用高級提示執行FL任務。我們觀察到，基於LLM的自動化解決方案在測試準確性方面與標準基於Web的解決方案相當，同時減少了高達64%的傳輸字節和高達46%的CPU時間用於FL任務。此外，我們利用LLM進行神經架構搜索（NAS）和超參數優化（HPO）以提高性能。我們觀察到，通過使用這種方法，測試準確性可以提高10-20%用於執行的FL任務。

English

Federated Learning (FL) offers a promising approach for collaborative machine learning across distributed devices. However, its adoption is hindered by the complexity of building reliable communication architectures and the need for expertise in both machine learning and network programming. This paper presents a comprehensive solution that simplifies the orchestration of FL tasks while integrating intent-based automation. We develop a user-friendly web application supporting the federated averaging (FedAvg) algorithm, enabling users to configure parameters through an intuitive interface. The backend solution efficiently manages communication between the parameter server and edge nodes. We also implement model compression and scheduling algorithms to optimize FL performance. Furthermore, we explore intent-based automation in FL using a fine-tuned Language Model (LLM) trained on a tailored dataset, allowing users to conduct FL tasks using high-level prompts. We observe that the LLM-based automated solution achieves comparable test accuracy to the standard web-based solution while reducing transferred bytes by up to 64% and CPU time by up to 46% for FL tasks. Also, we leverage the neural architecture search (NAS) and hyperparameter optimization (HPO) using LLM to improve the performance. We observe that by using this approach test accuracy can be improved by 10-20% for the carried out FL tasks.