AtomiMed：面向通用临床感知医学报告评估的分层原子事实核查

摘要

傳統醫學報告生成（MRG）的評估指標主要依賴表層的n-gram重疊，這種方式無法捕捉臨床事實的準確性，且經常忽略災難性的診斷錯誤。為了解決這一根本局限，我們提出AtomiMed——一個通用且與模態無關的評估框架，能將複雜的醫學敘事分解為標準化的多層級原子臨床事實層次結構，涵蓋疾病級實體及包括位置、形態與嚴重程度在內的屬性級描述符。透過在真實報告與預測報告之間實施智能體交叉驗證循環，AtomiMed模擬了多位放射科醫生的同儕審查流程，以驗證臨床一致性，從而實現診斷檢測與描述準確性的解耦評估。為促進標準化評估，我們推出MRGEvalKit——一個用於自動化階層式提取的開源工具包，並整理出OmniMRG-Bench，一個涵蓋X光、CT、MRI及超音波的全面多模態基準。在多項專家標註的閱讀者研究中進行的大量實驗表明，相較於傳統指標及基於模型的指標，AtomiMed與人類放射科醫生判斷的相關性顯著更高。我們的代碼已發布在 https://github.com/Venn2336/MRGEvalkit。

English

Traditional metrics for Medical Report Generation (MRG) predominantly rely on surface-level n-gram overlap, which fails to capture clinical factual accuracy and often overlooks catastrophic diagnostic errors. We address this fundamental limitation by proposing AtomiMed, a universal, modality-agnostic evaluation framework that decomposes complex medical narratives into a standardized, multi-level hierarchy of Atomic Clinical Facts, encompassing Disease-level entities and Attribute-level descriptors, including location, morphology, and severity. By implementing an Agentic Cross-Verification loop between ground-truth and predicted reports, AtomiMed simulates a multi-radiologist peer-review process to verify clinical consistency, thus enabling the decoupled assessment of diagnostic detection and descriptive accuracy. To facilitate standardized evaluation, we introduce MRGEvalKit, an open-source toolkit for automated hierarchical extraction, and curate OmniMRG-Bench, a comprehensive multi-modal benchmark covering X-ray, CT, MRI, and Ultrasound. Extensive experiments on multiple expert-annotated reader studies demonstrate that AtomiMed achieves significantly higher correlation with human radiologist judgment compared to traditional and model-based metrics. Our code are release at https://github.com/Venn2336/MRGEvalkit