PACEvolve：实现长周期进度感知的一致性演化

摘要

大型语言模型（LLMs）已成为进化搜索的强大执行者，但高效搜索框架的设计仍处于零散状态。尽管现有融入LLM的循环系统展现出潜力，但其进化过程管理缺乏系统性方法。我们识别出三种典型失效模式：情境污染（实验历史偏差影响后续候选生成）、模式坍塌（智能体因探索-利用失衡陷入局部最优）以及弱协作（僵化的交叉策略无法有效利用并行搜索轨迹）。为此，我们提出进度感知一致进化框架（PACEvolve），通过稳健控制智能体情境与搜索动态来应对这些挑战。该框架融合层级情境管理（HCM）与剪枝机制解决情境污染问题；采用动量回溯（MBB）逃离局部最优；通过自适应采样策略统一回溯与交叉操作，实现动态搜索协调（CE），使智能体能在内部优化与跨轨迹协作间取得平衡。实验表明，PACEvolve为持续长周期自我优化提供了系统化路径，在LLM-SR和KernelBench基准测试中达到最优水平，并在Modded NanoGPT任务上发现了超越现有记录的解决方案。

English

Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.

PACEvolve：实现长周期进度感知的一致性演化

PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution

摘要

Support