演化微调：跨371个优化任务学习发现

摘要

设计更快的GPU内核经验是否也有助于攻克一个长期悬而未决的数学猜想？整合了大型语言模型（LLM）的进化搜索已能在优化任务中产生最先进的解决方案，涵盖开放数学猜想、GPU内核设计、科学定律发现和组合谜题等领域。为此，先前的研究每次仅针对单一目标任务应用搜索框架，这意味着每个新问题都需从零开始探索，当模型完成尝试后，搜索过程中积累的经验便被丢弃。这使得迭代优化解决方案的能力（例如判断需修改的部分及修改方式、决定何时回溯）完全取决于框架本身，而非模型自身。模型能否自主掌握这种能力并在不同任务间复用，此前尚未得到充分探究。为解决这一问题，我们提出进化微调（EFT）——一种通过将进化搜索轨迹转化为监督信号，教导LLM跨任务优化解决方案的中期训练范式。我们构建了Finch集合数据集，包含覆盖10个领域和371个优化任务的15.6万条轨迹，并对20亿至90亿参数的开源LLM进行微调。实证表明，EFT实现了跨任务泛化：在22个保留任务中，我们的模型平均超越基础模型10.22%。进一步结合测试时强化学习后，我们的模型在两个圆形填充任务上达到最先进性能，并在埃尔德什最小重叠问题上超越基础模型对应版本。因此，EFT可作为通用型发现代理的“练习阶段”，而无需从零开始解决新问题。

English

Would experience designing faster GPU kernels also help close in on a long-standing open mathematical conjecture? Large Language Models (LLMs) integrated into evolutionary search have recently produced state-of-the-art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt. This leaves the capability of iteratively evolving a solution (e.g., knowing which part to mutate and how, deciding when to backtrack) entirely in the scaffold rather than in the model itself. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined. To address this, we introduce Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision. We construct Finch Collection, a 156K-trajectory dataset spanning 10 domains and 371 optimization tasks, and fine-tune open-source LLMs from 2B to 9B parameters. Empirically, EFT confers cross-task generalization: across 22 held-out tasks, our models surpass their base counterparts by 10.22% on average. Furthermore, when paired with test-time RL, our model matches state-of-the-art performance on two circle-packing tasks and outperforms its base-model counterpart on the Erdős minimum-overlap problem. EFT thus serves as a "practice phase" for general-purpose discovery agents that do not solve new problems from scratch.