图原生强化学习通过概念重组实现可追溯的科学假设生成

摘要

加速材料发现需要能够通过多步骤、基于领域的推理生成科学有效假设的AI系统。标准大语言模型在应对开放性材料设计问题时，往往生成流畅但可追溯性较弱的回答，使得难以判断最终答案是否得到连贯中间推理的支持。我们开发了Graph-PRefLexOR，一系列图原生推理模型，采用组相对策略优化（GRPO）进行微调，将推理过程组织为明确的阶段：机制探索、图构建、模式提取和假设综合。这种设计将神经语言生成与符号关系结构相结合，使得因果连接可以构建、检查并复用。在来自材料科学和力学文献的100个开放性问题上，Graph-PRefLexOR相较于对应基础模型实现了40%至65%的提升，其中推理可追溯性的改进最为显著。嵌入分析表明，其语义探索范围更广，语义多样性约为基线的2至3倍。语义回溯和逐层隐藏状态分析进一步显示，结构化推理与最终答案之间具有更强的对齐性。最后，测试时的图扩展表明，额外计算主要增加有界语义空间内的长程概念重组，而非简单扩大语义覆盖范围。这些结果确立了图原生强化学习作为通往可解释AI系统的路径，适用于材料设计及其他科学应用中的科学假设生成。

English

Accelerating materials discovery requires AI systems that can generate scientifically valid hypotheses through multi-step, domain-grounded reasoning. Standard large language models often produce fluent but weakly traceable responses to open-ended materials design problems, making it difficult to determine whether final answers are supported by coherent intermediate reasoning. We develop Graph-PRefLexOR, a family of graph-native reasoning models fine-tuned with Group Relative Policy Optimization (GRPO) to organize reasoning into explicit phases for mechanism exploration, graph construction, pattern extraction, and hypothesis synthesis. This design links neural language generation with symbolic relational structure, enabling causal connections to be constructed, inspected, and reused. On 100 open-ended questions from materials science and mechanics literature, Graph-PRefLexOR achieves 40-65% improvements over corresponding base models, with the largest gains in reasoning traceability. Embedding analyses show broader semantic exploration and approximately 2-3 times greater semantic diversity than baselines. Semantic backtracking and layer-wise hidden-state analyses further show stronger alignment between structured reasoning and final answers. Finally, test-time graph expansion reveals that additional compute primarily increases long-range conceptual recombination within a bounded semantic space, rather than simply expanding semantic coverage. These results establish graph-native reinforcement learning as a pathway toward interpretable AI systems for scientific hypothesis generation in materials design and other scientific applications.