牙科GPT：激勵牙科領域的多模態複雜推理

摘要

在口腔醫療自動化領域，多模態數據的可靠解讀至關重要，然而現有的多模態大語言模型（MLLMs）難以捕捉細粒度的牙科視覺細節，且缺乏精準診斷所需的充分推理能力。為解決這些局限，我們提出DentalGPT——通過高質量領域知識注入與強化學習開發的專業牙科MLLM。具體而言，我們整合了逾12萬張牙科影像及其標註診斷相關視覺特徵的詳細描述，構建了迄今規模最大的註釋多模態牙科數據集，這也是當前涵蓋最全面牙科影像的多模態數據集。基於此數據集的訓練顯著增強了MLLM對牙科病況的視覺理解能力，而後續的強化學習階段進一步強化了其多模態複雜推理效能。在口內影像與全景影像基準測試，以及醫學視覺問答（VQA）基準的牙科子集上的綜合評估表明，DentalGPT在疾病分類與牙科VQA任務中表現卓越，僅憑70億參數即超越多項先進MLLMs。這些結果證實，高質量牙科數據結合分階段適應策略，能為構建高效能的領域專用牙科MLLM提供有效路徑。

English

Reliable interpretation of multimodal data in dentistry is essential for automated oral healthcare, yet current multimodal large language models (MLLMs) struggle to capture fine-grained dental visual details and lack sufficient reasoning ability for precise diagnosis. To address these limitations, we present DentalGPT, a specialized dental MLLM developed through high-quality domain knowledge injection and reinforcement learning. Specifically, the largest annotated multimodal dataset for dentistry to date was constructed by aggregating over 120k dental images paired with detailed descriptions that highlight diagnostically relevant visual features, making it the multimodal dataset with the most extensive collection of dental images to date. Training on this dataset significantly enhances the MLLM's visual understanding of dental conditions, while the subsequent reinforcement learning stage further strengthens its capability for multimodal complex reasoning. Comprehensive evaluations on intraoral and panoramic benchmarks, along with dental subsets of medical VQA benchmarks, show that DentalGPT achieves superior performance in disease classification and dental VQA tasks, outperforming many state-of-the-art MLLMs despite having only 7B parameters. These results demonstrate that high-quality dental data combined with staged adaptation provides an effective pathway for building capable and domain-specialized dental MLLMs.

牙科GPT：激勵牙科領域的多模態複雜推理

DentalGPT: Incentivizing Multimodal Complex Reasoning in Dentistry

摘要

Support