论PEFT的规模化：迈向拥有万亿参数的百万个人模型

On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters

June 1, 2026

作者: Mind Lab, Song Cao, Vic Cao, Kaijie Chen, Bunny Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Hongquan Gu, Aaron Guan, Mutian Hong, Hailee Hou, Peixuan Hua, Charles Huang, Miles Jiang, Nora Jiang, Yuyi Jiang, Autumn Jin, Fancy Kong, Kyrie Lei, Alexy Li, Dawn Li, Ray Li, Theo Li, Wenhao Li, Jiayi Lin, Domini Liu, Heshan Liu, Kairus Liu, Logan Liu, Maeve Luo, Runism Lv, Pony Ma, Verity Niu, Anson Qiu, Vincent Wang, Maxwell Yao, Regis Ye, Wenlin Ye, Yanying Ye, Josh Ying, Danney Zeng, Salmon Zhan, Anya Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhang, Wei Zhao, Ada Zhou, Sizer Zhou, Xinyue Zhu, Murphy Zhuang

cs.AI

摘要

参数高效微调（PEFT）通常被视为全参数微调的经济替代方案。本研究探讨其更广泛的应用：将小型可训练适配器作为持久性局部状态，叠加在强大的共享基础模型之上。在此框架下，基础模型提供共享能力，而适配器承载实例特定行为，如偏好、技能、工具使用习惯及类似记忆的更新。我们围绕三个扩展维度组织问题：向上扩展——更强的共享先验使小型局部更新更具价值；向下扩展——研究适配器在保持可靠性的前提下能达到的最小规模；向外扩展——多个持久化适配实例共存。MinT提供了一个管理适配器身份、版本、溯源、评估及运行时驻留的基础设施范例。综合结果表明，PEFT可以作为持久化个性模型的紧凑基座，而不仅仅是全参数微调的预算替代品。

English

Parameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning.