PraMem：基於實踐的經驗記憶用於長時域行為預測

摘要

長期行為預測旨在基於長序列歷史推斷用戶的下一步行動，在人工智慧領域扮演關鍵角色。大型語言模型的興起為序列行為預測提供了極具前景的方向，然而在處理長期行為預測時，大型語言模型面臨潛在行為模式歸納困難以及模型內在認知偏差等挑戰。先前的記憶管理方法遵循情境壓縮範式，試圖透過減輕歷史序列負擔來解決此任務，但未能根治核心問題。本文倡導範式轉移，將冗長歷史序列從負擔重新定義為可開發的寶貴資源，並提出PraMem方法——通過對長歷史序列進行預先練習來建立經驗記憶，作為輔助輸入以實現精準的長期行為預測。跨多樣任務的廣泛實驗表明，PraMem較先前方法展現出更優異的性能，而深入分析更為經驗記憶的機制與演化提供了寶貴洞見。程式碼：https://github.com/icip-cas/PraMem。

English

Long-horizon behavior prediction aims to infer a user's next action based on a lengthy historical sequence, playing a crucial role in artificial intelligence field. The rise of large language models (LLMs) offers a promising direction for sequential behavior prediction, yet LLMs struggle with latent behavioral pattern induction and model-intrinsic cognitive biases when tackling long-horizon behavior prediction. Prior memory management methods follow a context-compression paradigm that attempts to address this task by alleviating the historical sequence burden, yet fail to resolve the core challenges. In this paper, we advocate a paradigm shift that reframes the lengthy historical sequence from a burden into a valuable resource to be exploited, and accordingly propose PraMem, which conducts beforehand practice over the lengthy historical sequence to build an experiential memory, thereby serving as the assisted input for accurate long-horizon behavior prediction. Extensive experiments across diverse tasks demonstrate that PraMem achieves superior performance than prior methods, and more in-depth analyses provide valuable insights into the mechanism and evolution of the experiential memory. Code: https://github.com/icip-cas/PraMem.