Fed-SE：面向隐私受限多环境大语言模型智能体的联邦自进化框架

摘要

尽管大语言模型智能体已广泛应用于复杂交互任务，但隐私约束往往阻碍其在动态环境中的集中式优化与协同进化。联邦学习虽在静态数据集上成效显著，但其在智能体开放式自主进化场景的扩展研究尚不充分。直接应用标准联邦学习面临挑战：异构任务特性与稀疏的轨迹级奖励会引发严重梯度冲突，导致全局优化过程失稳。为此，我们提出Fed-SE框架——一种面向大语言模型智能体的联邦自进化范式。该框架构建"本地进化-全局聚合"双层级机制：在本地层面，智能体基于筛选的高回报轨迹进行参数高效微调，实现稳定梯度更新；在全局层面，通过低秩子空间解耦环境特异性动态特征，有效聚合客户端更新以降低负迁移效应。在五个异构环境中的实验表明，Fed-SE相较联邦学习基线平均任务成功率提升约18%，验证了其在隐私约束部署下实现跨环境知识鲁棒迁移的有效性。

English

LLM agents are widely deployed in complex interactive tasks, yet privacy constraints often preclude centralized optimization and co-evolution across dynamic environments. While Federated Learning (FL) has proven effective on static datasets, its extension to the open-ended self-evolution of agents remains underexplored. Directly applying standard FL is challenging: heterogeneous tasks and sparse, trajectory-level rewards introduce severe gradient conflicts, destabilizing the global optimization process. To bridge this gap, we propose Fed-SE, a Federated Self-Evolution framework for LLM agents. Fed-SE establishes a local evolution-global aggregation paradigm. Locally, agents employ parameter-efficient fine-tuning on filtered, high-return trajectories to achieve stable gradient updates. Globally, Fed-SE aggregates updates within a low-rank subspace that disentangles environment-specific dynamics, effectively reducing negative transfer across clients. Experiments across five heterogeneous environments demonstrate that Fed-SE improves average task success rates by approximately 18% over federated baselines, validating its effectiveness in robust cross-environment knowledge transfer in privacy-constrained deployments.

Fed-SE：面向隐私受限多环境大语言模型智能体的联邦自进化框架

Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents

摘要

Support