PraMem: 基于实践的经验记忆用于长期行为预测

摘要

长程行为预测旨在基于长序列历史记录推断用户的下一步行为，在人工智能领域具有关键作用。大语言模型（LLMs）的兴起为序列行为预测提供了有前景的方向，但在处理长程行为预测时，LLMs在潜在行为模式归纳和模型固有认知偏差方面仍面临挑战。先前的记忆管理方法遵循上下文压缩范式，试图通过减轻历史序列负担来解决该任务，但未能解决核心难题。本文倡导一种范式转变，将长序列历史记录从负担重新定义为有待挖掘的宝贵资源，并据此提出PraMem方法，该方法预先对长序列历史记录进行练习，构建经验记忆，从而作为辅助输入实现精准的长程行为预测。跨不同任务的大量实验表明，PraMem比先前方法取得了更优性能，更深入的分析为经验记忆的机制和演化提供了宝贵见解。代码链接：https://github.com/icip-cas/PraMem。

English

Long-horizon behavior prediction aims to infer a user's next action based on a lengthy historical sequence, playing a crucial role in artificial intelligence field. The rise of large language models (LLMs) offers a promising direction for sequential behavior prediction, yet LLMs struggle with latent behavioral pattern induction and model-intrinsic cognitive biases when tackling long-horizon behavior prediction. Prior memory management methods follow a context-compression paradigm that attempts to address this task by alleviating the historical sequence burden, yet fail to resolve the core challenges. In this paper, we advocate a paradigm shift that reframes the lengthy historical sequence from a burden into a valuable resource to be exploited, and accordingly propose PraMem, which conducts beforehand practice over the lengthy historical sequence to build an experiential memory, thereby serving as the assisted input for accurate long-horizon behavior prediction. Extensive experiments across diverse tasks demonstrate that PraMem achieves superior performance than prior methods, and more in-depth analyses provide valuable insights into the mechanism and evolution of the experiential memory. Code: https://github.com/icip-cas/PraMem.