基于超图记忆的长上下文复杂关系建模多步RAG优化
Improving Multi-step RAG with Hypergraph-based Memory for Long-Context Complex Relational Modeling
December 30, 2025
作者: Chulun Zhou, Chunkang Zhang, Guoxin Yu, Fandong Meng, Jie Zhou, Wai Lam, Mo Yu
cs.AI
摘要
多步检索增强生成（RAG）已成为提升大语言模型在需要全局理解和深度推理任务上性能的广泛策略。现有RAG系统常通过工作记忆模块整合检索信息，但这类记忆设计多作为被动存储器，仅用于累积孤立事实以压缩长输入或通过推导生成新子查询。这种静态特性忽略原始事实间关键的高阶关联——而此类关联组合往往能为后续步骤提供更强指引。因此，现有方法在表征能力及对多步推理和知识演化的影响方面存在局限，导致长上下文中的推理碎片化与全局语义理解能力薄弱。
我们提出HGMem，一种基于超图的记忆机制，将记忆概念从简单存储扩展为支持复杂推理和全局理解的动态表达结构。该方法将记忆表征为超图，其超边对应独立记忆单元，可实现记忆中高阶交互的渐进式构建。该机制围绕核心问题连接事实与思维，逐步演变为集成化的情境知识结构，为后续步骤的深度推理提供强命题支持。我们在多个专为全局语义理解设计的挑战性数据集上评估HGMem。大量实验与深度分析表明，该方法能持续优化多步RAG性能，并在多样任务中显著超越强基线系统。
English
Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large language models (LLMs) on tasks that demand global comprehension and intensive reasoning. Many RAG systems incorporate a working memory module to consolidate retrieved information. However, existing memory designs function primarily as passive storage that accumulates isolated facts for the purpose of condensing the lengthy inputs and generating new sub-queries through deduction. This static nature overlooks the crucial high-order correlations among primitive facts, the compositions of which can often provide stronger guidance for subsequent steps. Therefore, their representational strength and impact on multi-step reasoning and knowledge evolution are limited, resulting in fragmented reasoning and weak global sense-making capacity in extended contexts. We introduce HGMem, a hypergraph-based memory mechanism that extends the concept of memory beyond simple storage into a dynamic, expressive structure for complex reasoning and global understanding. In our approach, memory is represented as a hypergraph whose hyperedges correspond to distinct memory units, enabling the progressive formation of higher-order interactions within memory. This mechanism connects facts and thoughts around the focal problem, evolving into an integrated and situated knowledge structure that provides strong propositions for deeper reasoning in subsequent steps. We evaluate HGMem on several challenging datasets designed for global sense-making. Extensive experiments and in-depth analyses show that our method consistently improves multi-step RAG and substantially outperforms strong baseline systems across diverse tasks.