DoctorAgent-RL：一种面向多轮临床对话的多智能体协作强化学习系统

DoctorAgent-RL: A Multi-Agent Collaborative Reinforcement Learning System for Multi-Turn Clinical Dialogue

May 26, 2025

作者: Yichun Feng, Jiawei Wang, Lu Zhou, Yixue Li

cs.AI

摘要

大型语言模型（LLMs）在生物医学问答领域展现了卓越的能力，但其在实际临床咨询中的应用仍面临核心挑战。现有系统依赖单向信息传递模式，患者需一次性完整描述症状，导致当主诉模糊时，诊断建议缺乏针对性。基于监督学习的传统多轮对话方法受限于静态数据驱动范式，泛化能力不足，难以智能提取关键临床信息。为克服这些局限，我们提出了DoctorAgent-RL，一个基于强化学习（RL）的多智能体协作框架，将医疗咨询建模为不确定性下的动态决策过程。医生智能体在RL框架内通过与患者智能体的多轮交互，持续优化其提问策略，并根据咨询评估器提供的综合奖励动态调整信息收集路径。这一RL微调机制使LLMs能够自主开发符合临床推理逻辑的交互策略，而非浅层模仿现有对话数据中的模式。值得注意的是，我们构建了MTMedDialog，首个能够模拟患者互动的英文多轮医疗咨询数据集。实验表明，DoctorAgent-RL在多轮推理能力和最终诊断性能上均优于现有模型，展现了在辅助临床咨询中的实用价值。https://github.com/JarvisUSTC/DoctorAgent-RL

English

Large language models (LLMs) have demonstrated excellent capabilities in the field of biomedical question answering, but their application in real-world clinical consultations still faces core challenges. Existing systems rely on a one-way information transmission mode where patients must fully describe their symptoms in a single round, leading to nonspecific diagnostic recommendations when complaints are vague. Traditional multi-turn dialogue methods based on supervised learning are constrained by static data-driven paradigms, lacking generalizability and struggling to intelligently extract key clinical information. To address these limitations, we propose DoctorAgent-RL, a reinforcement learning (RL)-based multi-agent collaborative framework that models medical consultations as a dynamic decision-making process under uncertainty. The doctor agent continuously optimizes its questioning strategy within the RL framework through multi-turn interactions with the patient agent, dynamically adjusting its information-gathering path based on comprehensive rewards from the Consultation Evaluator. This RL fine-tuning mechanism enables LLMs to autonomously develop interaction strategies aligned with clinical reasoning logic, rather than superficially imitating patterns in existing dialogue data. Notably, we constructed MTMedDialog, the first English multi-turn medical consultation dataset capable of simulating patient interactions. Experiments demonstrate that DoctorAgent-RL outperforms existing models in both multi-turn reasoning capability and final diagnostic performance, demonstrating practical value in assisting clinical consultations. https://github.com/JarvisUSTC/DoctorAgent-RL