基于超图记忆的长上下文复杂关系建模增强多步检索增强生成

Improving Multi-step RAG with Hypergraph-based Memory for Long-Context Complex Relational Modeling

December 30, 2025

作者: Chulun Zhou, Chunkang Zhang, Guoxin Yu, Fandong Meng, Jie Zhou, Wai Lam, Mo Yu

cs.AI

摘要

多步检索增强生成（RAG）已成为增强大语言模型在需要全局理解和深度推理任务上表现的广泛策略。现有RAG系统多采用工作记忆模块来整合检索信息，但传统记忆设计主要作为被动存储器，通过累积孤立事实来实现长输入压缩和演绎生成新子查询。这种静态特性忽略了原始事实间关键的高阶关联，而此类关联组合往往能为后续步骤提供更强指引。因此，现有方法在表征能力和对多步推理及知识演进的影响方面存在局限，导致长上下文中的推理碎片化和全局语义构建能力薄弱。我们提出HGMem——一种基于超图的记忆机制，将记忆概念从简单存储扩展为支持复杂推理和全局理解的动态表达结构。该机制将记忆表示为超图，其超边对应不同记忆单元，可实现记忆内高阶交互的渐进式形成。这种设计围绕核心问题连接事实与思维，逐步演化为集成化、情境化的知识结构，为后续步骤的深度推理提供强命题支持。我们在多个专为全局语义构建设计的挑战性数据集上评估HGMem。大量实验与深度分析表明，该方法能持续优化多步RAG性能，并在多样化任务中显著超越强基线系统。

English

Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large language models (LLMs) on tasks that demand global comprehension and intensive reasoning. Many RAG systems incorporate a working memory module to consolidate retrieved information. However, existing memory designs function primarily as passive storage that accumulates isolated facts for the purpose of condensing the lengthy inputs and generating new sub-queries through deduction. This static nature overlooks the crucial high-order correlations among primitive facts, the compositions of which can often provide stronger guidance for subsequent steps. Therefore, their representational strength and impact on multi-step reasoning and knowledge evolution are limited, resulting in fragmented reasoning and weak global sense-making capacity in extended contexts. We introduce HGMem, a hypergraph-based memory mechanism that extends the concept of memory beyond simple storage into a dynamic, expressive structure for complex reasoning and global understanding. In our approach, memory is represented as a hypergraph whose hyperedges correspond to distinct memory units, enabling the progressive formation of higher-order interactions within memory. This mechanism connects facts and thoughts around the focal problem, evolving into an integrated and situated knowledge structure that provides strong propositions for deeper reasoning in subsequent steps. We evaluate HGMem on several challenging datasets designed for global sense-making. Extensive experiments and in-depth analyses show that our method consistently improves multi-step RAG and substantially outperforms strong baseline systems across diverse tasks.