管理LLM智能体中的程序性记忆：控制、适应与评估

摘要

程序性记忆正被越来越多地用于提升大语言模型智能体在重复性工作场景中的表现，但其生成可复用技能的能力仍未被充分理解。我们提出AFTER基准，包含382个真实企业任务，覆盖六种职业角色和22项程序性技能，旨在评估技能在任务、角色和模型主干间的迁移能力。该基准设计了可控评估设置，涵盖局部提升、跨任务迁移、跨角色迁移和跨模型泛化。实验表明，程序性记忆在工业工作流中能带来持续增益：单次优化使整体性能提升3.7-6.7个百分点，而基于多样化多模型执行轨迹演化的技能实现了73.1%的跨模型测试准确率，优于所有单一模型轨迹来源。我们进一步发现，部分技能在任务和模型间广泛泛化，而另一些则专精于特定角色工作流，并在迁移时失去有效性。这些结果为在生产化智能体平台上构建、评估和部署程序性记忆系统提供了实践指导。

English

Procedural memory is increasingly used to improve LLM agents on recurring workplace tasks, yet its ability to produce reusable skills remains poorly understood. We introduce AFTER, a benchmark of 382 realistic enterprise tasks spanning six professional roles and 22 procedural skills, designed to evaluate how skills transfer across tasks, roles, and model backbones. The benchmark includes controlled evaluation settings for local improvement, cross-task transfer, cross-role transfer, and cross-model generalization. Experiments show that procedural memory delivers consistent gains in industrial workflows: a single refinement round improves aggregate performance by 3.7-6.7 points, while skills evolved from diverse multi-model execution traces achieve 73.1% cross-model test accuracy, outperforming all single-model trace sources. We further find that some skills generalize broadly across tasks and models, whereas others become specialized to role-specific workflows and lose effectiveness under transfer. These results provide practical guidance for building, evaluating, and deploying procedural memory systems in production agent platforms.