大型發現模型：以實證為基礎的基於模型開放式搜尋

摘要

科學發現通常涉及在廣闊、結構化且開放式的假設空間中優化評估成本高昂的目標，例如分子、蛋白質序列和電腦程式。諸如大型語言模型（LLMs）之類的生成模型為這類空間提供了表達力強的先驗，但其似然和自我評估對於目標及校準的認知不確定性而言，是不可靠的代理指標，尤其是對於超出觀測數據分佈的新候選項。我們引入了大型發現模型（LDM），這是一種基於經驗的循環架構，將生成模型與貝葉斯非參數獎勵代理模型結合在一起。生成模型提出並改進候選設計，而代理模型則預測其性能並量化不確定性，從而產生一個具不確定性感知的價值，用以引導候選項的生成、改進和選擇。發現記憶和代理模型會隨著每一次新的實驗觀測的到來而不斷更新。我們在跨越不同設計模式和目標的三種情境中評估LDM，包括神經網路訓練、抗體設計和分子優化。與僅使用LLM的反思或傳統統計搜索相比，LDM在這些領域中實現了驗證BPB額外2.4倍的降低，結合能相對降低18.2%，以及分子多目標性能超過60%的相對增益。這些結果表明，LDM可作為一種通用發現引擎，用於在開放式假設空間中進行有效搜索。

English

Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation. Compared to LLM-only reflection or traditional statistical search across these domains, LDM achieves a 2.4times greater reduction in validation BPB, an 18.2% relative decrease in binding energy, and more than 60% relative gains in molecular multi-objective performance. These results suggests that LDM could serve as a general-purpose discovery engine for effective search over open-ended hypothesis spaces.