Mergenetic：一個簡易的演化模型融合庫

Mergenetic: a Simple Evolutionary Model Merging Library

May 16, 2025

作者: Adrian Robert Minut, Tommaso Mencattini, Andrea Santilli, Donato Crisostomi, Emanuele Rodolà

cs.AI

摘要

模型融合技術允許將現有模型的能力結合到一個新模型中——無需額外訓練，即可事後完成。這項技術因其低成本及支援消費級GPU融合的函式庫普及而日益流行。近期研究顯示，將融合與進化算法配對可提升性能，但目前尚無框架支援在語言模型中靈活實驗此類策略。我們介紹了Mergenetic，一個用於進化模型融合的開源函式庫。Mergenetic不僅簡化了融合方法與進化算法的組合，還整合了輕量級適應度評估器以降低評估成本。我們闡述了其設計，並展示Mergenetic在適度硬體條件下，跨任務與語言均能產出競爭力十足的成果。

English

Model merging allows combining the capabilities of existing models into a new one - post hoc, without additional training. This has made it increasingly popular thanks to its low cost and the availability of libraries that support merging on consumer GPUs. Recent work shows that pairing merging with evolutionary algorithms can boost performance, but no framework currently supports flexible experimentation with such strategies in language models. We introduce Mergenetic, an open-source library for evolutionary model merging. Mergenetic enables easy composition of merging methods and evolutionary algorithms while incorporating lightweight fitness estimators to reduce evaluation costs. We describe its design and demonstrate that Mergenetic produces competitive results across tasks and languages using modest hardware.