HealthAgentBench：面向挑战前沿AI智能体的统一逼真智能体医疗环境基准测试套件

摘要

随着AI代理在复杂长程推理方面的能力不断增强，对其开展严格且全面的评估对于衡量其在真实医疗场景中的进展至关重要。我们推出HealthAgentBench，该基准包含54项医疗代理任务，涵盖7个类别，每个类别拥有独特的环境。这一基准套件覆盖了患者诊疗全流程中的多样化工作流以及广泛的模态。每项任务都旨在复现端到端的临床工作流：代理仅凭极简指令，需探索原始医疗数据，在复杂环境中操作，并执行超越简单提示的多步骤解决方案。最终报告任务成功率，为HealthAgentBench上各代理的整体表现提供一个单一、可解读的指标。在HealthAgentBench上评估前沿代理时，我们发现整体任务成功率仍然较低，凸显了该套件的难度。性能最强且最具成本效益的代理Codex GPT-5.5，其成功率仅为约42%。除了整体表现，HealthAgentBench还揭示了各任务类别中细微的优势与不足。前沿代理在基于电子健康记录数据自动开发研究建模流程方面展现出潜力，但医学影像处理仍然尤其具有挑战性，尤其对Claude Code模型而言，而Codex GPT-5.5则展现出初步能力。结合大搜索空间与组合推理要求的任务对所有当前代理来说仍然困难。这些结果共同表明，HealthAgentBench提供了一个具有挑战性且贴近现实的基准，为未来进步留出了充足空间。我们已在https://github.com/microsoft/HealthAgentBench上发布该基准。

English

As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment. The benchmark suite spans diverse workflows throughout the patient journey and a broad range of modalities. Each task is designed to replicate an end-to-end clinical workflow: given minimal instructions, an agent must explore raw healthcare data, operate within a complex environment, and execute multi-step solutions that go beyond naive prompting. A final task success rate is reported to provide a single, interpretable metric for HealthAgentBench overall performance for each agent. Evaluating frontier agents on HealthAgentBench, we find that overall task success rate remains low, underscoring the difficulty of the suite. The strongest and the most cost effective agent, Codex GPT-5.5, achieves only approximately 42% success rate. Beyond aggregate performance, HealthAgentBench reveals nuanced strengths and weaknesses across task categories. Frontier agents show promise in automatically developing research modeling pipelines over EHR data, but medical imaging remains especially challenging, particularly for Claude Code models, while Codex GPT-5.5 shows emerging capability. Tasks that combine large search spaces with compositional reasoning requirements remain difficult for all current agents. Together, these results suggest that HealthAgentBench provides a challenging and realistic benchmark with substantial room for future progress. We release our benchmark at https://github.com/microsoft/HealthAgentBench.