邁向通用生物醫學人工智慧

摘要

醫學本質上是多模態的，具有豐富的數據形式，包括文本、影像、基因組等。靈活編碼、整合和解釋這些數據的通用生物醫學人工智慧（AI）系統，潛在地可以實現從科學發現到護理交付等具有影響力的應用。為了促進這些模型的開發，我們首先編輯了MultiMedBench，這是一個新的多模態生物醫學基準。MultiMedBench 包含了14個不同的任務，如醫學問答、乳房X光攝影和皮膚科影像解釋、放射學報告生成和摘要，以及基因組變異呼叫等。然後，我們介紹了Med-PaLM Multimodal（Med-PaLM M），這是我們的一個通用生物醫學AI系統的概念證明。Med-PaLM M 是一個大型多模態生成模型，可以靈活編碼和解釋包括臨床語言、影像和基因組等在內的生物醫學數據，使用相同的模型權重。Med-PaLM M 在所有MultiMedBench任務上達到了與或超越最先進技術的競爭性表現，往往超過專家模型很大範圍。我們還報告了對新醫學概念和任務的零-shot泛化示例，跨任務的正向轉移學習，以及新興的零-shot醫學推理。為了進一步探討Med-PaLM M 的能力和局限性，我們對模型生成（和人類）胸部X光報告進行了放射學家評估，觀察到在不同模型規模下表現令人鼓舞。在對246例回顧性胸部X光進行並排排名時，臨床醫生在多達40.50%的情況下對Med-PaLM M 的報告表現出兩兩偏好，這表明潛在的臨床效用。雖然需要大量工作來驗證這些模型在實際用例中的有效性，但我們的結果代表了通用生物醫學AI系統發展的一個里程碑。

English

Medicine is inherently multimodal, with rich data modalities spanning text, imaging, genomics, and more. Generalist biomedical artificial intelligence (AI) systems that flexibly encode, integrate, and interpret this data at scale can potentially enable impactful applications ranging from scientific discovery to care delivery. To enable the development of these models, we first curate MultiMedBench, a new multimodal biomedical benchmark. MultiMedBench encompasses 14 diverse tasks such as medical question answering, mammography and dermatology image interpretation, radiology report generation and summarization, and genomic variant calling. We then introduce Med-PaLM Multimodal (Med-PaLM M), our proof of concept for a generalist biomedical AI system. Med-PaLM M is a large multimodal generative model that flexibly encodes and interprets biomedical data including clinical language, imaging, and genomics with the same set of model weights. Med-PaLM M reaches performance competitive with or exceeding the state of the art on all MultiMedBench tasks, often surpassing specialist models by a wide margin. We also report examples of zero-shot generalization to novel medical concepts and tasks, positive transfer learning across tasks, and emergent zero-shot medical reasoning. To further probe the capabilities and limitations of Med-PaLM M, we conduct a radiologist evaluation of model-generated (and human) chest X-ray reports and observe encouraging performance across model scales. In a side-by-side ranking on 246 retrospective chest X-rays, clinicians express a pairwise preference for Med-PaLM M reports over those produced by radiologists in up to 40.50% of cases, suggesting potential clinical utility. While considerable work is needed to validate these models in real-world use cases, our results represent a milestone towards the development of generalist biomedical AI systems.

邁向通用生物醫學人工智慧

Towards Generalist Biomedical AI

摘要

Support