演化微調：在371個優化任務中學習發現

摘要

設計更快GPU核心的經驗，是否也有助於攻克長期懸而未決的數學猜想？近期，結合大型語言模型（LLM）與演化搜索的方法，已在最佳化任務中取得最先進的解決方案，涵蓋了未解數學猜想、GPU核心設計、科學定律發現及組合謎題等領域。為達成此目標，先前研究每次僅針對單一目標任務應用搜索框架，每遇新問題便從頭開始，而模型在搜索過程中所累積的經驗，一旦完成嘗試便遭捨棄。這使得反覆迭代改進解方（例如判斷哪些部分應突變及如何突變、決定何時回溯）的能力完全由框架承擔，而非模型本身。模型是否能自身習得此能力並跨不同任務重複使用，至今仍鮮少被探討。為填補此缺口，我們提出「演化微調」（Evolution Fine-Tuning，EFT）——一種中期訓練範式，透過將演化搜索軌跡轉化為監督訊號，教導LLM跨任務演化解決方案。我們建構了「Finch Collection」資料集，涵蓋10個領域共371項最佳化任務的156,000條軌跡，並對2B至9B參數規模的開源LLM進行微調。實驗結果顯示，EFT賦予模型跨任務泛化能力：在22項保留任務中，我們的模型平均超越其基礎版本10.22%。此外，搭配測試時強化學習（test-time RL）時，我們的模型在兩個圓形填充任務中達到與最先進水準相當的表現，並在Erdős最小重疊問題上超越其基礎模型版本。因此，EFT可視為通用發現代理程式的「練習階段」，使其無須從頭解決新問題。

English

Would experience designing faster GPU kernels also help close in on a long-standing open mathematical conjecture? Large Language Models (LLMs) integrated into evolutionary search have recently produced state-of-the-art solutions on optimization tasks, including open mathematical conjectures, GPU kernel design, scientific law discovery, and combinatorial puzzles. To achieve this, prior work applied search scaffolds to one target task at a time, so every new problem is approached from scratch and the experience accumulated during search is discarded once the model finishes its attempt. This leaves the capability of iteratively evolving a solution (e.g., knowing which part to mutate and how, deciding when to backtrack) entirely in the scaffold rather than in the model itself. Whether the model itself could acquire this capability and reuse it across different tasks has been largely unexamined. To address this, we introduce Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across tasks by converting evolutionary search trajectories into supervision. We construct Finch Collection, a 156K-trajectory dataset spanning 10 domains and 371 optimization tasks, and fine-tune open-source LLMs from 2B to 9B parameters. Empirically, EFT confers cross-task generalization: across 22 held-out tasks, our models surpass their base counterparts by 10.22% on average. Furthermore, when paired with test-time RL, our model matches state-of-the-art performance on two circle-packing tasks and outperforms its base-model counterpart on the Erdős minimum-overlap problem. EFT thus serves as a "practice phase" for general-purpose discovery agents that do not solve new problems from scratch.