管理LLM代理中的程序性記憶：控制、適應與評估

摘要

程序性記憶正日益被用於提升LLM智能體在重複性工作任務中的表現，然而其能否產出可複用技能仍未被充分理解。我們提出AFTER基準，內含382項涵蓋六種專業角色與22項程序性技能的現實企業任務，旨在評估技能如何跨任務、跨角色及跨模型主幹進行遷移。該基準包含針對局部改進、跨任務遷移、跨角色遷移及跨模型泛化的受控評估設置。實驗顯示，程序性記憶能夠在工業工作流中帶來持續增益：單次優化輪次即可將總體性能提升3.7至6.7個百分點，而從多種多模型執行軌跡演化而來的技能，其跨模型測試準確率達73.1%，優於所有單模型軌跡來源。我們進一步發現，部分技能可在任務與模型間廣泛泛化，另一些則專門化於特定角色工作流，並在遷移中失效。這些結果為在生產級智能體平台上構建、評估及部署程序性記憶系統提供了實務指引。

English

Procedural memory is increasingly used to improve LLM agents on recurring workplace tasks, yet its ability to produce reusable skills remains poorly understood. We introduce AFTER, a benchmark of 382 realistic enterprise tasks spanning six professional roles and 22 procedural skills, designed to evaluate how skills transfer across tasks, roles, and model backbones. The benchmark includes controlled evaluation settings for local improvement, cross-task transfer, cross-role transfer, and cross-model generalization. Experiments show that procedural memory delivers consistent gains in industrial workflows: a single refinement round improves aggregate performance by 3.7-6.7 points, while skills evolved from diverse multi-model execution traces achieve 73.1% cross-model test accuracy, outperforming all single-model trace sources. We further find that some skills generalize broadly across tasks and models, whereas others become specialized to role-specific workflows and lose effectiveness under transfer. These results provide practical guidance for building, evaluating, and deploying procedural memory systems in production agent platforms.