Mergenetic：一个简洁的进化模型融合库

Mergenetic: a Simple Evolutionary Model Merging Library

May 16, 2025

作者: Adrian Robert Minut, Tommaso Mencattini, Andrea Santilli, Donato Crisostomi, Emanuele Rodolà

cs.AI

摘要

模型融合技术使得无需额外训练即可将现有模型的能力整合到一个新模型中，这一后处理方式因其低成本及支持消费级GPU融合的库的普及而日益流行。近期研究表明，将融合技术与进化算法结合可提升性能，但目前尚无框架支持在语言模型中灵活尝试此类策略。为此，我们推出了Mergenetic，一个用于进化模型融合的开源库。Mergenetic不仅简化了融合方法与进化算法的组合，还引入了轻量级适应度评估器以降低评估成本。我们详述了其设计理念，并通过实验证明，Mergenetic在多种任务和语言上均能利用普通硬件取得具有竞争力的成果。

English

Model merging allows combining the capabilities of existing models into a new one - post hoc, without additional training. This has made it increasingly popular thanks to its low cost and the availability of libraries that support merging on consumer GPUs. Recent work shows that pairing merging with evolutionary algorithms can boost performance, but no framework currently supports flexible experimentation with such strategies in language models. We introduce Mergenetic, an open-source library for evolutionary model merging. Mergenetic enables easy composition of merging methods and evolutionary algorithms while incorporating lightweight fitness estimators to reduce evaluation costs. We describe its design and demonstrate that Mergenetic produces competitive results across tasks and languages using modest hardware.