關於參數高效微調的規模化：邁向百萬個擁有萬億參數的個人模型

On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters

June 1, 2026

作者: Mind Lab, Song Cao, Vic Cao, Kaijie Chen, Bunny Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Hongquan Gu, Aaron Guan, Mutian Hong, Hailee Hou, Peixuan Hua, Charles Huang, Miles Jiang, Nora Jiang, Yuyi Jiang, Autumn Jin, Fancy Kong, Kyrie Lei, Alexy Li, Dawn Li, Ray Li, Theo Li, Wenhao Li, Jiayi Lin, Domini Liu, Heshan Liu, Kairus Liu, Logan Liu, Maeve Luo, Runism Lv, Pony Ma, Verity Niu, Anson Qiu, Vincent Wang, Maxwell Yao, Regis Ye, Wenlin Ye, Yanying Ye, Josh Ying, Danney Zeng, Salmon Zhan, Anya Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhang, Wei Zhao, Ada Zhou, Sizer Zhou, Xinyue Zhu, Murphy Zhuang

cs.AI

摘要

參數高效微調（PEFT）通常被視為完整微調的一種更便宜的替代方案。我們研究一個更廣泛的角色：小型可訓練適配器作為在強大共享基礎模型之上的持久局部狀態。在此框架下，基礎模型提供共享能力，而適配器承載實例特定的行為，例如偏好、技能、工具習慣，以及類似記憶的更新。我們圍繞三個擴展軸來組織問題：向上擴展——更強的共享先驗使得小型局部更新變得更有用；向下擴展——我們研究適配器在保持可靠的同時可以小到何種程度；向外擴展——許多持久化的適配實例共存。MinT 提供了一個基礎設施範例，用於管理適配器的身份、修訂、溯源、評估及服務駐留。綜合而言，這些結果表明，PEFT 可以成為持久化個人模型的緊湊基礎，而不僅僅是完整微調的預算替代方案。

English

Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning.