蘆薈家族打造開放與專業化醫療大語言模型的秘方

The Aloe Family Recipe for Open and Specialized Healthcare LLMs

May 7, 2025

作者: Dario Garcia-Gasulla, Jordi Bayarri-Planas, Ashwin Kumar Gururajan, Enrique Lopez-Cuena, Adrian Tormos, Daniel Hinjos, Pablo Bernabeu-Perez, Anna Arias-Duart, Pablo Agustin Martin-Torres, Marta Gonzalez-Mallo, Sergio Alvarez-Napagao, Eduard Ayguadé-Parra, Ulises Cortés

cs.AI

摘要

目的：随着大型语言模型（LLMs）在医疗领域的进步，开发具有竞争力的开源模型以保护公众利益的需求日益凸显。本研究通过优化数据预处理和训练的关键阶段，展示了如何通过直接偏好优化（DPO）提升模型安全性，以及通过检索增强生成（RAG）提高模型效能，为开源医疗LLM领域做出了贡献。所采用的评估方法，包括四种不同类型的测试，为该领域设定了新的标准。最终发布的模型在性能上与最佳私有替代品相当，并以宽松的许可协议发布。 方法：基于Llama 3.1和Qwen 2.5等强大基础模型，Aloe Beta利用定制数据集，通过合成思维链示例增强公共数据。模型经过直接偏好优化对齐，强调在遭遇越狱攻击时的伦理与政策一致性表现。评估包括封闭式、开放式、安全性和人类评估，以最大化结果的可靠性。 结果：基于Aloe系列模型的坚实表现，提出了贯穿整个流程的优化建议。这些模型在医疗基准和医学领域中展现出竞争力，并常受医疗专业人士青睐。在偏见和毒性方面，Aloe Beta模型显著提升了安全性，对未知越狱攻击表现出韧性。为负责任地发布，Aloe系列模型附有详细的医疗风险评估。 结论：Aloe Beta模型及其开发方法，为开源医疗LLM领域做出了重要贡献，提供了顶尖性能的同时，坚守了高伦理要求。本研究为医疗领域对齐LLM的开发与报告设定了新标准。

English

Purpose: With advancements in Large Language Models (LLMs) for healthcare, the need arises for competitive open-source models to protect the public interest. This work contributes to the field of open medical LLMs by optimizing key stages of data preprocessing and training, while showing how to improve model safety (through DPO) and efficacy (through RAG). The evaluation methodology used, which includes four different types of tests, defines a new standard for the field. The resultant models, shown to be competitive with the best private alternatives, are released with a permisive license. Methods: Building on top of strong base models like Llama 3.1 and Qwen 2.5, Aloe Beta uses a custom dataset to enhance public data with synthetic Chain of Thought examples. The models undergo alignment with Direct Preference Optimization, emphasizing ethical and policy-aligned performance in the presence of jailbreaking attacks. Evaluation includes close-ended, open-ended, safety and human assessments, to maximize the reliability of results. Results: Recommendations are made across the entire pipeline, backed by the solid performance of the Aloe Family. These models deliver competitive performance across healthcare benchmarks and medical fields, and are often preferred by healthcare professionals. On bias and toxicity, the Aloe Beta models significantly improve safety, showing resilience to unseen jailbreaking attacks. For a responsible release, a detailed risk assessment specific to healthcare is attached to the Aloe Family models. Conclusion: The Aloe Beta models, and the recipe that leads to them, are a significant contribution to the open-source medical LLM field, offering top-of-the-line performance while maintaining high ethical requirements. This work sets a new standard for developing and reporting aligned LLMs in healthcare.