大语言模型代理强化学习研究全景：综述

摘要

代理强化学习（Agentic RL）的兴起标志着从传统应用于大语言模型（LLM RL）的强化学习向一种新范式的转变，将大语言模型从被动的序列生成器重新定义为嵌入复杂动态世界中的自主决策代理。本综述通过对比LLM-RL中简化的单步马尔可夫决策过程（MDPs）与定义Agentic RL的时间扩展、部分可观测的马尔可夫决策过程（POMDPs），正式阐述了这一概念转变。基于此，我们提出了一个全面的双重分类体系：一类围绕核心代理能力组织，包括规划、工具使用、记忆、推理、自我改进和感知；另一类则围绕这些能力在多样化任务领域中的应用展开。我们的核心论点是，强化学习是将这些能力从静态的启发式模块转化为适应性强、鲁棒的代理行为的关键机制。为了支持和加速未来研究，我们将开源环境、基准测试和框架的现状整合成一份实用指南。通过综合五百余篇近期文献，本综述勾勒了这一快速发展领域的轮廓，并强调了将塑造可扩展、通用人工智能代理发展的机遇与挑战。

English

The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm shift from conventional reinforcement learning applied to large language models (LLM RL), reframing LLMs from passive sequence generators into autonomous, decision-making agents embedded in complex, dynamic worlds. This survey formalizes this conceptual shift by contrasting the degenerate single-step Markov Decision Processes (MDPs) of LLM-RL with the temporally extended, partially observable Markov decision processes (POMDPs) that define Agentic RL. Building on this foundation, we propose a comprehensive twofold taxonomy: one organized around core agentic capabilities, including planning, tool use, memory, reasoning, self-improvement, and perception, and the other around their applications across diverse task domains. Central to our thesis is that reinforcement learning serves as the critical mechanism for transforming these capabilities from static, heuristic modules into adaptive, robust agentic behavior. To support and accelerate future research, we consolidate the landscape of open-source environments, benchmarks, and frameworks into a practical compendium. By synthesizing over five hundred recent works, this survey charts the contours of this rapidly evolving field and highlights the opportunities and challenges that will shape the development of scalable, general-purpose AI agents.

大语言模型代理强化学习研究全景：综述

The Landscape of Agentic Reinforcement Learning for LLMs: A Survey

摘要

Support