LingxiDiagBench：一個用於在中文精神科諮詢與診斷中對大型語言模型進行基準測試的多智能體框架

摘要

精神障礙在全球範圍內盛行率極高，但精神科醫師的短缺以及基於面談診斷固有的主觀性，對及時且一致的心理健康評估造成了重大障礙。人工智慧輔助精神科診斷的進展受限於缺乏同時提供逼真的患者模擬、臨床醫師驗證的診斷標籤、並支援動態多輪問診的基準測試。我們提出 LingxiDiagBench，一個大規模的多智能體基準測試，旨在評估大型語言模型在中文環境下進行靜態診斷推理與動態多輪精神科問診的能力。其核心是 LingxiDiag-16K，一個包含 16,000 份與電子病歷對齊的合成問診對話資料集，旨在重現涵蓋 12 種 ICD-10 精神科類別的實際臨床人口統計與診斷分佈。透過對當前頂尖大型語言模型的廣泛實驗，我們確立了以下關鍵發現：（1）儘管大型語言模型在二元憂鬱-焦慮分類上達到高準確率（最高 92.3%），但在憂鬱-焦慮共病識別（43.0%）與 12 類鑑別診斷（28.5%）的表現卻顯著下降；（2）動態問診的表現往往不如靜態評估，顯示無效的資訊收集策略會顯著損害後續的診斷推理能力；（3）以「LLM作為評審」評估的問診品質與診斷準確率之間僅呈中度相關，這表明結構良好的提問並不能確保正確的診斷決策。我們已釋出 LingxiDiag-16K 與完整的評估框架，以支援可重複研究，網址為 https://github.com/Lingxi-mental-health/LingxiDiagBench。

English

Mental disorders are highly prevalent worldwide, but the shortage of psychiatrists and the inherent subjectivity of interview-based diagnosis create substantial barriers to timely and consistent mental-health assessment. Progress in AI-assisted psychiatric diagnosis is constrained by the absence of benchmarks that simultaneously provide realistic patient simulation, clinician-verified diagnostic labels, and support for dynamic multi-turn consultation. We present LingxiDiagBench, a large-scale multi-agent benchmark that evaluates LLMs on both static diagnostic inference and dynamic multi-turn psychiatric consultation in Chinese. At its core is LingxiDiag-16K, a dataset of 16,000 EMR-aligned synthetic consultation dialogues designed to reproduce real clinical demographic and diagnostic distributions across 12 ICD-10 psychiatric categories. Through extensive experiments across state-of-the-art LLMs, we establish key findings: (1) although LLMs achieve high accuracy on binary depression--anxiety classification (up to 92.3%), performance deteriorates substantially for depression--anxiety comorbidity recognition (43.0%) and 12-way differential diagnosis (28.5%); (2) dynamic consultation often underperforms static evaluation, indicating that ineffective information-gathering strategies significantly impair downstream diagnostic reasoning; (3) consultation quality assessed by LLM-as-a-Judge shows only moderate correlation with diagnostic accuracy, suggesting that well-structured questioning alone does not ensure correct diagnostic decisions. We release LingxiDiag-16K and the full evaluation framework to support reproducible research at https://github.com/Lingxi-mental-health/LingxiDiagBench.