圖原生強化學習透過概念重組實現可追溯的科學假設生成

摘要

加速材料發現需要能透過多步驟、基於領域的推理來生成科學有效假設的AI系統。標準的大型語言模型在面對開放式材料設計問題時，通常會產出流暢但難以追溯的回應，使判斷最終答案是否奠基於連貫的中間推理變得困難。我們開發了Graph-PRefLexOR，這是一系列圖原生推理模型，透過群組相對策略最佳化（GRPO）進行微調，將推理組織為明確的階段：機制探索、圖形建構、模式提取與假設綜合。此設計將神經語言生成與符號關係結構連結，使因果關係得以建構、檢視及重用。在來自材料科學與力學文獻的100個開放式問題上，Graph-PRefLexOR相較對應基礎模型提升了40-65%，其中推理可追溯性的增益最大。嵌入分析顯示，其語義探索範圍更廣，且語義多樣性約為基線的2-3倍。語義回溯與逐層隱藏狀態分析進一步表明，結構化推理與最終答案之間有更強的對齊。最後，測試時圖形擴展揭示，增加的計算資源主要促進有限語義空間內長程概念的重組，而非單純擴展語義覆蓋範圍。這些結果確立了圖原生強化學習作為通往可解釋AI系統的途徑，適用於材料設計及其他科學應用中的科學假設生成。

English

Accelerating materials discovery requires AI systems that can generate scientifically valid hypotheses through multi-step, domain-grounded reasoning. Standard large language models often produce fluent but weakly traceable responses to open-ended materials design problems, making it difficult to determine whether final answers are supported by coherent intermediate reasoning. We develop Graph-PRefLexOR, a family of graph-native reasoning models fine-tuned with Group Relative Policy Optimization (GRPO) to organize reasoning into explicit phases for mechanism exploration, graph construction, pattern extraction, and hypothesis synthesis. This design links neural language generation with symbolic relational structure, enabling causal connections to be constructed, inspected, and reused. On 100 open-ended questions from materials science and mechanics literature, Graph-PRefLexOR achieves 40-65% improvements over corresponding base models, with the largest gains in reasoning traceability. Embedding analyses show broader semantic exploration and approximately 2-3 times greater semantic diversity than baselines. Semantic backtracking and layer-wise hidden-state analyses further show stronger alignment between structured reasoning and final answers. Finally, test-time graph expansion reveals that additional compute primarily increases long-range conceptual recombination within a bounded semantic space, rather than simply expanding semantic coverage. These results establish graph-native reinforcement learning as a pathway toward interpretable AI systems for scientific hypothesis generation in materials design and other scientific applications.