大型发现模型：基于实证的模型驱动开放式搜索

摘要

科学发现通常涉及在庞大、结构化且开放式的假设空间（如分子、蛋白质序列和计算机程序）中对评估代价高昂的目标进行优化。生成模型（如大型语言模型，LLM）为这类空间提供了富有表现力的先验，但其似然度和自我评估作为目标函数及经过校准的认知不确定性的代理指标并不可靠，尤其是对于观测数据分布之外的新颖候选方案。我们提出了大型发现模型（LDM），一种基于经验构建的循环架构，将生成模型与贝叶斯非参数奖励代理模型相结合。生成模型负责提出并优化候选设计，而代理模型则预测其性能并量化不确定性，从而产生一种具有不确定性感知的价值，用以指导候选方案的生成、优化和筛选。随着每一条新实验观测数据的到来，发现记忆和代理模型都会持续更新。我们在三种涵盖不同设计模态和目标的场景中评估了LDM，包括神经网络训练、抗体设计和分子优化。与仅依赖LLM反思或传统统计搜索的方法相比，LDM在这些领域中将验证BPB的降幅提升了2.4倍，结合能相对降低了18.2%，分子多目标性能相对提升了60%以上。这些结果表明，LDM可以作为在开放式假设空间中进行高效搜索的通用发现引擎。

English

Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation. Compared to LLM-only reflection or traditional statistical search across these domains, LDM achieves a 2.4times greater reduction in validation BPB, an 18.2% relative decrease in binding energy, and more than 60% relative gains in molecular multi-objective performance. These results suggests that LDM could serve as a general-purpose discovery engine for effective search over open-ended hypothesis spaces.