离散扩散语言模型在交互式放射学报告起草中的应用

摘要

扩散语言模型通过双向去噪令牌画布（而非从左到右逐个生成令牌）来生成文本，现已与自回归（AR）生成能力相匹敌。然而，医学基础模型几乎仍完全采用自回归架构。我们适配了一个混合专家扩散语言模型 DiffusionGemma-26B，在视觉问答医学数据集上，采用相同的 LoRA 配置，将其与同等规模的 AR 模型 Gemma-4-26B 进行基准测试，并由对冗余度稳健的大语言模型评判器评分。在所有数据集上，扩散模型的表现均持平或优于自回归模型；该微调模型（有效参数 3.8B）可与前沿视觉语言模型相竞争，且其解码速度也快 3.5 至 4.4 倍。除性能持平外，扩散模型还具备自回归模型所欠缺的起草能力：任意顺序填充。由于画布通过双向去噪处理，放射科医生可以固定报告片段，让模型填充文字片段之间的部分；这一操作是扩散模型的固有功能，而自回归模型对此表现欠佳。这恰好适用于真实的临床报告，此类报告往往因临床医生或机构不同而简洁或不一致。

English

Diffusion language models, which generate text by denoising a token canvas bidirectionally instead of emitting tokens left to right, have become competitive with autoregressive (AR) generation. Medical foundation models, however, remain almost entirely autoregressive. We adapt a mixture-of-experts diffusion language model, DiffusionGemma-26B, and benchmark it against its same-size AR sibling Gemma-4-26B under an identical LoRA recipe on medical visual question answering datasets, scored by a verbosity-robust LLM judge. Diffusion matches or exceeds AR on all of them, and the finetuned model (3.8B active) is competitive with frontier vision-language models; its decoding is also 3.5-4.4x faster. Beyond this parity, the diffusion model offers a drafting capability AR lacks: any-order infill. Because the canvas is denoised bidirectionally, a radiologist can fix report fragments and have the model fill the text between them, an operation inherent to diffusion but not to autoregression, which is subpar at it. This suits real reports, which are often terse or inconsistent across clinicians and institutions.