PACEvolve：实现长程进度感知一致性演进的框架

摘要

大型语言模型（LLMs）已成为进化搜索的强大执行者，但高效搜索框架的设计仍处于零散状态。尽管现有LLM参与循环的系统展现出潜力，却缺乏管理进化过程的系统性方法。我们识别出三种典型失效模式：语境污染（实验历史偏差影响后续候选生成）、模式坍塌（智能体因探索-利用失衡陷入局部最优）、以及弱协作（僵化的交叉策略无法有效利用并行搜索轨迹）。为此，我们提出进展感知一致进化框架（PACEvolve），通过稳健管控智能体的语境与搜索动态来解决这些挑战。该框架融合层级化语境管理（HCM）与剪枝策略应对语境污染；采用动量回溯（MBB）机制逃离局部最优；并通过自适应采样策略统一回溯与交叉操作，实现动态搜索协调（CE），使智能体能平衡内部优化与跨轨迹协作。实验表明，PACEvolve为持续长程自我改进提供了系统化路径，在LLM-SR和KernelBench基准上达到顶尖水平，并在Modded NanoGPT任务上发现了超越现有记录的解决方案。

English

Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.

PACEvolve：实现长程进度感知一致性演进的框架

PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution

摘要

Support