貝葉斯優化的自適應核設計，借助大型語言模型，如同享用一塊蛋糕般輕鬆

摘要

貝葉斯優化（BO）的效率在很大程度上依賴於高斯過程（GP）核的選擇，該核在有限的評估預算下對探索與利用的平衡起著核心作用。傳統的BO方法通常依賴於固定或啟發式的核選擇策略，當所選核與底層目標函數不匹配時，可能導致收斂緩慢或次優解。為解決這一限制，我們提出了一種新穎的上下文感知核演化（CAKE）方法，利用大型語言模型（LLMs）來增強BO。具體而言，CAKE利用LLMs作為交叉和變異算子，根據優化過程中觀察到的數據自適應地生成和精煉GP核。為了最大化CAKE的效能，我們進一步提出了基於貝葉斯信息準則（BIC）的獲取核排序（BAKER）方法，通過平衡BIC衡量的模型擬合度與每次BO迭代的期望改進來選擇最有效的核。大量實驗表明，我們基於CAKE的新BO方法在一系列實際任務中，包括超參數優化、控制器調諧和光子芯片設計，均顯著優於現有的基準方法。我們的代碼已公開於https://github.com/cake4bo/cake。

English

The efficiency of Bayesian optimization (BO) relies heavily on the choice of the Gaussian process (GP) kernel, which plays a central role in balancing exploration and exploitation under limited evaluation budgets. Traditional BO methods often rely on fixed or heuristic kernel selection strategies, which can result in slow convergence or suboptimal solutions when the chosen kernel is poorly suited to the underlying objective function. To address this limitation, we propose a freshly-baked Context-Aware Kernel Evolution (CAKE) to enhance BO with large language models (LLMs). Concretely, CAKE leverages LLMs as the crossover and mutation operators to adaptively generate and refine GP kernels based on the observed data throughout the optimization process. To maximize the power of CAKE, we further propose BIC-Acquisition Kernel Ranking (BAKER) to select the most effective kernel through balancing the model fit measured by the Bayesian information criterion (BIC) with the expected improvement at each iteration of BO. Extensive experiments demonstrate that our fresh CAKE-based BO method consistently outperforms established baselines across a range of real-world tasks, including hyperparameter optimization, controller tuning, and photonic chip design. Our code is publicly available at https://github.com/cake4bo/cake.

貝葉斯優化的自適應核設計，借助大型語言模型，如同享用一塊蛋糕般輕鬆

Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs

摘要

Support