ArogyaSutra：一种面向印度语言的多模态医疗推理多智能体框架

摘要

多模态大语言模型（MLLMs）在通用领域展现出了有前景的推理能力，但在医疗健康等专业场景中性能仍然有限，尤其是在多语言和低资源情境下。这一差距在农村印度等地区尤为关键：患者常以本土印度语言表达复杂的医疗问题，并依赖医学影像等多模态输入。现有以英语为中心的MLLMs难以支持此类应用场景，限制了人们公平获取人工智能驱动的医疗辅助服务。为应对这一挑战，我们提出了ArogyaBodha——一个大规模多语言多模态医学问答数据集，它整合了八个异构来源，覆盖31个身体系统、六种成像模态及21个临床领域，涵盖英语和七种主要印度语言。我们还进一步提出了ArogyaSutra——一个基于演员-评论家（actor-critic）的多智能体框架，该框架将工具调用与双记忆机制相结合，实现逐步的、推理感知的决策过程，并利用存储的演员-评论家模拟轨迹进行知识蒸馏。实验表明，我们的数据集和框架在所有印度语言上均提升了多语言医学推理的准确性，消融实验验证了各组件的贡献。源代码和数据集已发布在：https://iitp-cse.github.io/ArogyaSutra/

English

Multimodal Large Language Models (MLLMs) have shown promising reasoning capabilities in general domains, yet their performance remains limited in specialized settings such as healthcare, especially in multilingual and low-resource scenarios. This gap is critical in regions like rural India, where patients often express complex medical queries in native Indic languages and rely on multimodal inputs such as medical images. Existing English-centric MLLMs struggle to support such use cases, limiting equitable access to AI-driven healthcare assistance. To address this challenge, we introduce ArogyaBodha, a large-scale multilingual multimodal medical question-answer dataset constructed from eight heterogeneous sources, covering 31 body systems, six imaging modalities, and 21 clinical domains across English and seven major Indian languages. We further propose ArogyaSutra, an actor-critic-based multi-agent framework that integrates tool grounding with dual-memory mechanisms for step-wise, reasoning-aware decision making, and uses stored actor-critic simulation trajectories for distillation. Experiments show that our dataset and framework improve multilingual medical reasoning accuracy across all Indic languages, with ablations validating the contribution of each component. The source code and dataset are available at: https://iitp-cse.github.io/ ArogyaSutra/