MedMobile：具有专业水平临床能力的移动尺寸语言模型

MedMobile: A mobile-sized language model with expert-level clinical capabilities

October 11, 2024

作者: Krithik Vishwanath, Jaden Stryker, Anton Alaykin, Daniel Alexander Alber, Eric Karl Oermann

cs.AI

摘要

语言模型（LMs）已经展示了在医学领域专家级的推理和回忆能力。然而，计算成本和隐私问题正在成为广泛实施的障碍。我们介绍了phi-3-mini的简约适应版本MedMobile，这是一个拥有38亿参数的LM，可以在移动设备上运行，用于医学应用。我们展示了MedMobile在MedQA（USMLE）上获得了75.7%的分数，超过了医生的及格线（约60%），接近于其100倍大小模型的分数。随后，我们进行了一系列仔细的消融实验，并展示了思维链、集成和微调带来了最大的性能提升，而意外的检索增强生成未能显示出显著的改进。

English

Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-scale implementation. We introduce a parsimonious adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of running on a mobile device, for medical applications. We demonstrate that MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for physicians (~60%), and approaching the scores of models 100 times its size. We subsequently perform a careful set of ablations, and demonstrate that chain of thought, ensembling, and fine-tuning lead to the greatest performance gains, while unexpectedly retrieval augmented generation fails to demonstrate significant improvements