微調小型語言模型以實現領域專用AI：從邊緣AI的視角探討

Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective

March 3, 2025

作者: Rakshit Aralimatti, Syed Abdul Gaffar Shakhadri, Kruthika KR, Kartik Basavaraj Angadi

cs.AI

摘要

在邊緣設備上部署大規模語言模型面臨著固有的挑戰，如高計算需求、能源消耗以及潛在的數據隱私風險。本文介紹了針對這些限制直接應對的Shakti小型語言模型（SLMs）系列——Shakti-100M、Shakti-250M和Shakti-500M。通過結合高效的架構、量化技術及負責任的人工智能原則，Shakti系列為智能手機、智能家電、物聯網系統等提供了設備端智能。我們深入探討了其設計理念、訓練流程，以及在通用任務（如MMLU、Hellaswag）和專業領域（醫療、金融、法律）上的基準性能。研究結果表明，經過精心設計和微調的緊湊模型，在實際的邊緣AI場景中不僅能滿足甚至常常超越預期。

English

Deploying large scale language models on edge devices faces inherent challenges such as high computational demands, energy consumption, and potential data privacy risks. This paper introduces the Shakti Small Language Models (SLMs) Shakti-100M, Shakti-250M, and Shakti-500M which target these constraints headon. By combining efficient architectures, quantization techniques, and responsible AI principles, the Shakti series enables on-device intelligence for smartphones, smart appliances, IoT systems, and beyond. We provide comprehensive insights into their design philosophy, training pipelines, and benchmark performance on both general tasks (e.g., MMLU, Hellaswag) and specialized domains (healthcare, finance, and legal). Our findings illustrate that compact models, when carefully engineered and fine-tuned, can meet and often exceed expectations in real-world edge-AI scenarios.