Agent-Reflex：教会语言智能体进行反思性行动

Agent-Reflex: Teaching Language Agents to Act Reflectively

October 13, 2026

作者: Guanting Dong, Xinyu Yang, Jiajie Jin, Yuyang Hu, Xiaoshuai Song, Yifei Chen, Xiaoxi Li, Jiejun Tan, Yutao Zhu, Tat-Seng Chua, Ji-Rong Wen, Zhicheng Dou

cs.AI

摘要

大语言模型已从单轮聊天机器人演变为能够进行长时程推理的交互式智能体。在实际部署中，MCP 工具服务可能返回噪声反馈，用户也可能介入纠正智能体，但现有方法对反思学习和失败恢复提供的监督有限。为弥补这一差距，我们提出 Agent-Reflex，一种用于激励智能体主动反思的通用训练框架。具体而言，Agent-Reflex 包含三个阶段。首先，环境反思技能发现（Environment Reflection Skill Discovery, RSD）在环境扩展中注入基于真实工具反馈的扰动，采样能力多样的大语言模型池，挖掘匹配的成功-失败对，并总结可复用的反思元技能。基于该技能库，我们随后应用反思式监督微调（Reflective Supervised Fine-Tuning, ReSFT），将每项技能转化为对自我反思、技能恢复和以用户为中心的纠正的监督信号，从而强化互补的反思行为。最后，我们提出同策略反思策略优化（On-Policy Reflective Policy Optimization, ORPO），将噪声交互下的智能体强化学习与反思引导的同策略蒸馏相结合，以进一步激发主动反思。基于 Qwen3-8B，Agent-Reflex 在五个长时程工具使用基准上达到 58.6 的总体得分，在 AgentNoiseBench 上达到 29.1，超越了强大的环境扩展与反思基线。在六个具有挑战性的智能体基准套件上的实验证明了 Agent-Reflex 的有效性。进一步分析表明，Agent-Reflex 提高了智能体对噪声的鲁棒性，同时保持稳定的训练动态，从而支持可扩展的智能体训练。我们的代码可在 https://github.com/RUC-NLPIR/Agent-Reflex 获取。

English

Large language models have evolved from single-turn chatbots into interactive agents capable of long-horizon reasoning. In real deployments, MCP tool services can return noisy feedback and users may intervene to correct the agent, but existing methods provide limited supervision for reflection learning and failure recovery. To bridge this gap, we propose Agent-Reflex, a general training framework for incentivizing proactive reflection in agents. Specifically, Agent-Reflex comprises three stages. First, Environment Reflection Skill Discovery (RSD) injects grounded tool-feedback perturbations into environment scaling, samples a capability-diverse LLM pool, mines matched success-failure pairs, and summarizes reusable reflection meta-skills. Building on this skill library, we then apply Reflective Supervised Fine-Tuning (ReSFT) that converts each skill into supervision for self-reflection, skill recovery, and user-centric correction, strengthening complementary reflective behaviors. Finally, we propose On-Policy Reflective Policy Optimization (ORPO) to combine agentic reinforcement learning under noisy interactions with reflection-guided on-policy distillation to further elicit proactive reflection. Built on Qwen3-8B, Agent-Reflex reaches 58.6 overall on five long-horizon tool-use benchmarks and 29.1 on AgentNoiseBench, surpassing strong environment-scaling and reflection baselines. Experiments on six challenging agent benchmark suites demonstrate the effectiveness of Agent-Reflex. Further analyses show that Agent-Reflex improves agents' robustness to noise while maintaining stable training dynamics, supporting scalable agent training. Our code is available at https://github.com/RUC-NLPIR/Agent-Reflex.