HeurAgenix：运用大语言模型攻克复杂组合优化难题

摘要

启发式算法在解决组合优化（CO）问题中扮演着至关重要的角色，然而传统设计高度依赖人工经验，难以在不同实例间实现泛化。我们提出了HeurAgenix，一个由大型语言模型（LLMs）驱动的两阶段超启发式框架，它首先进化启发式方法，随后自动从中进行选择。在启发式进化阶段，HeurAgenix利用LLM比较初始启发式解与更高质量的解，并提取可复用的进化策略。在问题求解过程中，它根据LLM的感知能力动态选取每个问题状态下最有前景的启发式方法。为了灵活性，这一选择器既可以是先进的LLM，也可以是经过微调、推理成本较低的轻量级模型。针对组合优化复杂性导致的可靠监督稀缺问题，我们采用双奖励机制微调轻量级启发式选择器，该机制联合利用选择偏好与状态感知的信号，确保在噪声标注下实现稳健选择。在经典基准测试上的广泛实验表明，HeurAgenix不仅超越了现有的基于LLM的超启发式方法，还达到或超越了专用求解器的性能。代码已发布于https://github.com/microsoft/HeurAgenix。

English

Heuristic algorithms play a vital role in solving combinatorial optimization (CO) problems, yet traditional designs depend heavily on manual expertise and struggle to generalize across diverse instances. We introduce HeurAgenix, a two-stage hyper-heuristic framework powered by large language models (LLMs) that first evolves heuristics and then selects among them automatically. In the heuristic evolution phase, HeurAgenix leverages an LLM to compare seed heuristic solutions with higher-quality solutions and extract reusable evolution strategies. During problem solving, it dynamically picks the most promising heuristic for each problem state, guided by the LLM's perception ability. For flexibility, this selector can be either a state-of-the-art LLM or a fine-tuned lightweight model with lower inference cost. To mitigate the scarcity of reliable supervision caused by CO complexity, we fine-tune the lightweight heuristic selector with a dual-reward mechanism that jointly exploits singals from selection preferences and state perception, enabling robust selection under noisy annotations. Extensive experiments on canonical benchmarks show that HeurAgenix not only outperforms existing LLM-based hyper-heuristics but also matches or exceeds specialized solvers. Code is available at https://github.com/microsoft/HeurAgenix.

HeurAgenix：运用大语言模型攻克复杂组合优化难题

HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial Optimization Challenges

摘要

Support