MOOSE-Chem2:透過階層式搜尋探索大型語言模型在細粒度科學假設發現中的極限
MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search
May 25, 2025
作者: Zonglin Yang, Wanhao Liu, Ben Gao, Yujie Liu, Wei Li, Tong Xie, Lidong Bing, Wanli Ouyang, Erik Cambria, Dongzhan Zhou
cs.AI
摘要
大型語言模型(LLMs)在自動化科學假設生成方面展現了潛力,然而現有方法主要產生的假設較為粗粒度,缺乏關鍵的方法論和實驗細節。我們引入並正式定義了細粒度科學假設發現這一新任務,該任務要求從粗略的初始研究方向生成詳細且可實驗操作的假設。我們將其框架為一個組合優化問題,並探討了在最大程度利用下,LLMs解決此問題的能力上限。具體而言,我們探討了四個基礎問題:(1) 如何最佳地利用LLM的內部啟發式來制定其自身認為在所有可能生成的假設中最有前景的細粒度假設,基於其內部評分——從而定義假設空間上的潛在獎勵景觀;(2) 此類由LLM評判的較好假設是否與真實假設表現出更強的對齊性;(3) 使用一組能力相似的多樣化LLMs來塑造獎勵景觀,是否比使用其中最強LLM的重複實例來定義獎勵景觀能產生更好的結果;以及(4) 一組相同的LLMs是否比單個LLM提供更可靠的獎勵景觀。為解決這些問題,我們提出了一種分層搜索方法,該方法逐步提出並將細節整合到假設中,從一般概念進展到具體的實驗配置。我們展示了這一分層過程平滑了獎勵景觀,並實現了更有效的優化。在一個基於近期化學文獻中專家註釋的細粒度假設的新基準上的實證評估表明,我們的方法始終優於強基線。
English
Large language models (LLMs) have shown promise in automating scientific
hypothesis generation, yet existing approaches primarily yield coarse-grained
hypotheses lacking critical methodological and experimental details. We
introduce and formally define the novel task of fine-grained scientific
hypothesis discovery, which entails generating detailed, experimentally
actionable hypotheses from coarse initial research directions. We frame this as
a combinatorial optimization problem and investigate the upper limits of LLMs'
capacity to solve it when maximally leveraged. Specifically, we explore four
foundational questions: (1) how to best harness an LLM's internal heuristics to
formulate the fine-grained hypothesis it itself would judge as the most
promising among all the possible hypotheses it might generate, based on its own
internal scoring-thus defining a latent reward landscape over the hypothesis
space; (2) whether such LLM-judged better hypotheses exhibit stronger alignment
with ground-truth hypotheses; (3) whether shaping the reward landscape using an
ensemble of diverse LLMs of similar capacity yields better outcomes than
defining it with repeated instances of the strongest LLM among them; and (4)
whether an ensemble of identical LLMs provides a more reliable reward landscape
than a single LLM. To address these questions, we propose a hierarchical search
method that incrementally proposes and integrates details into the hypothesis,
progressing from general concepts to specific experimental configurations. We
show that this hierarchical process smooths the reward landscape and enables
more effective optimization. Empirical evaluations on a new benchmark of
expert-annotated fine-grained hypotheses from recent chemistry literature show
that our method consistently outperforms strong baselines.Summary
AI-Generated Summary