关于先验的思考：大语言模型在知识图谱上的可信推理

Deliberation on Priors: Trustworthy Reasoning of Large Language Models on Knowledge Graphs

May 21, 2025

作者: Jie Ma, Ning Qu, Zhitao Gao, Rui Xing, Jun Liu, Hongbin Pei, Jiang Xie, Linyun Song, Pinghui Wang, Jing Tao, Zhou Su

cs.AI

摘要

基于知识图谱的检索增强生成旨在缓解大型语言模型（LLMs）因知识不足或过时而产生的幻觉问题。然而，现有方法往往未能充分利用知识图谱（KGs）中蕴含的先验知识，尤其是其结构信息及显式或隐式约束。前者可增强LLMs推理的忠实性，后者则能提升响应生成的可靠性。基于此，我们提出了一种可信推理框架，称为“先验审思”（Deliberation over Priors, DP），该框架充分挖掘了KGs中的先验知识。具体而言，DP采用渐进式知识蒸馏策略，通过结合监督微调与卡尼曼-特沃斯基优化，将结构先验融入LLMs，从而提升关系路径生成的忠实度。此外，我们的框架还运用了推理自省策略，引导LLMs基于提取的约束先验进行精细化推理验证，确保响应生成的可靠性。在三个基准数据集上的大量实验表明，DP实现了新的最先进性能，特别是在ComplexWebQuestions数据集上Hit@1指标提升了13%，并生成了高度可信的响应。我们还进行了多项分析，验证了其灵活性与实用性。代码已发布于https://github.com/reml-group/Deliberation-on-Priors。

English

Knowledge graph-based retrieval-augmented generation seeks to mitigate hallucinations in Large Language Models (LLMs) caused by insufficient or outdated knowledge. However, existing methods often fail to fully exploit the prior knowledge embedded in knowledge graphs (KGs), particularly their structural information and explicit or implicit constraints. The former can enhance the faithfulness of LLMs' reasoning, while the latter can improve the reliability of response generation. Motivated by these, we propose a trustworthy reasoning framework, termed Deliberation over Priors (DP), which sufficiently utilizes the priors contained in KGs. Specifically, DP adopts a progressive knowledge distillation strategy that integrates structural priors into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky optimization, thereby improving the faithfulness of relation path generation. Furthermore, our framework employs a reasoning-introspection strategy, which guides LLMs to perform refined reasoning verification based on extracted constraint priors, ensuring the reliability of response generation. Extensive experiments on three benchmark datasets demonstrate that DP achieves new state-of-the-art performance, especially a Hit@1 improvement of 13% on the ComplexWebQuestions dataset, and generates highly trustworthy responses. We also conduct various analyses to verify its flexibility and practicality. The code is available at https://github.com/reml-group/Deliberation-on-Priors.