Agent-Reflex：教導語言代理進行反思性行動

Agent-Reflex: Teaching Language Agents to Act Reflectively

October 13, 2026

作者: Guanting Dong, Xinyu Yang, Jiajie Jin, Yuyang Hu, Xiaoshuai Song, Yifei Chen, Xiaoxi Li, Jiejun Tan, Yutao Zhu, Tat-Seng Chua, Ji-Rong Wen, Zhicheng Dou

cs.AI

摘要

大型語言模型已從單輪聊天機器人演進為具備長程推理能力的互動式代理。在實際部署中，MCP 工具服務可能回傳雜訊回饋，且使用者可能介入以修正代理，但現有方法對反思學習與失敗復原提供的監督有限。為彌補此缺口，我們提出 Agent-Reflex，一個用於激勵代理主動反思的通用訓練框架。具體而言，Agent-Reflex 包含三個階段。首先，環境反思技能發現（RSD）將具環境依據的工具回饋擾動注入環境擴展，從能力多樣的 LLM 池中取樣，挖掘匹配的成功-失敗配對，並總結可重用的反思元技能。在此技能庫的基礎上，我們接著應用反思式監督微調（ReSFT），將每項技能轉換為自我反思、技能復原與以使用者為中心之修正的監督訊號，強化互補的反思行為。最後，我們提出同策略反思式策略最佳化（ORPO），將雜訊互動下的代理式強化學習與反思引導的同策略蒸餾結合，以進一步激發主動反思。Agent-Reflex 以 Qwen3-8B 為基礎，在五個長程工具使用基準上達到總體 58.6，並在 AgentNoiseBench 上達到 29.1，超越強大的環境擴展與反思基線。在六個具挑戰性的代理基準套件上的實驗證明了 Agent-Reflex 的有效性。進一步分析顯示，Agent-Reflex 提升代理對雜訊的穩健性，同時維持穩定的訓練動態，支持可擴展的代理訓練。我們的程式碼位於 https://github.com/RUC-NLPIR/Agent-Reflex。

English

Large language models have evolved from single-turn chatbots into interactive agents capable of long-horizon reasoning. In real deployments, MCP tool services can return noisy feedback and users may intervene to correct the agent, but existing methods provide limited supervision for reflection learning and failure recovery. To bridge this gap, we propose Agent-Reflex, a general training framework for incentivizing proactive reflection in agents. Specifically, Agent-Reflex comprises three stages. First, Environment Reflection Skill Discovery (RSD) injects grounded tool-feedback perturbations into environment scaling, samples a capability-diverse LLM pool, mines matched success-failure pairs, and summarizes reusable reflection meta-skills. Building on this skill library, we then apply Reflective Supervised Fine-Tuning (ReSFT) that converts each skill into supervision for self-reflection, skill recovery, and user-centric correction, strengthening complementary reflective behaviors. Finally, we propose On-Policy Reflective Policy Optimization (ORPO) to combine agentic reinforcement learning under noisy interactions with reflection-guided on-policy distillation to further elicit proactive reflection. Built on Qwen3-8B, Agent-Reflex reaches 58.6 overall on five long-horizon tool-use benchmarks and 29.1 on AgentNoiseBench, surpassing strong environment-scaling and reflection baselines. Experiments on six challenging agent benchmark suites demonstrate the effectiveness of Agent-Reflex. Further analyses show that Agent-Reflex improves agents' robustness to noise while maintaining stable training dynamics, supporting scalable agent training. Our code is available at https://github.com/RUC-NLPIR/Agent-Reflex.