模型合并配方的进化优化

Evolutionary Optimization of Model Merging Recipes

March 19, 2024

作者: Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, David Ha

cs.AI

摘要

我们提出了一种新颖的应用进化算法来自动化创建强大的基础模型。虽然模型合并已经成为LLM开发的一种有前途的方法，因为它具有成本效益，但目前仍然依赖人类直觉和领域知识，从而限制了其潜力。在这里，我们提出了一种进化方法，通过自动发现不同开源模型的有效组合，利用它们的集体智慧，而无需大量额外的训练数据或计算资源，从而克服了这一限制。我们的方法在参数空间和数据流空间中运作，允许进行超出单个模型权重的优化。这种方法甚至促进了跨领域合并，生成了具有数学推理能力的日语LLM等模型。令人惊讶的是，我们的日语数学LLM在各种已建立的日语LLM基准测试中取得了最先进的性能，甚至超过了具有更多参数的模型，尽管它们并未明确针对这些任务进行训练。此外，通过我们的方法生成的具有文化意识的日语VLM展示了其在描述日本文化特定内容方面的有效性，胜过了先前的日语VLM。这项工作不仅向开源社区贡献了新的最先进模型，还引入了一种新的自动化模型组合范式，为探索基础模型开发的替代高效方法铺平了道路。

English

We present a novel application of evolutionary algorithms to automate the creation of powerful foundation models. While model merging has emerged as a promising approach for LLM development due to its cost-effectiveness, it currently relies on human intuition and domain knowledge, limiting its potential. Here, we propose an evolutionary approach that overcomes this limitation by automatically discovering effective combinations of diverse open-source models, harnessing their collective intelligence without requiring extensive additional training data or compute. Our approach operates in both parameter space and data flow space, allowing for optimization beyond just the weights of the individual models. This approach even facilitates cross-domain merging, generating models like a Japanese LLM with Math reasoning capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with significantly more parameters, despite not being explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM generated through our approach demonstrates its effectiveness in describing Japanese culture-specific content, outperforming previous Japanese VLMs. This work not only contributes new state-of-the-art models back to the open-source community, but also introduces a new paradigm for automated model composition, paving the way for exploring alternative, efficient approaches to foundation model development.