TAT：面向多任务自适应的医学图像一体化复原Transformer模型

TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration

December 16, 2025

作者: Zhiwen Yang, Jiaju Zhang, Yang Yi, Jian Liang, Bingzheng Wei, Yan Xu

cs.AI

摘要

医学图像恢复（MedIR）旨在从低质量医学图像中重建高质量图像。当前MedIR研究聚焦于能同时处理多种恢复任务的一体化模型，但由于模态与退化类型存在显著差异，共享模型需重点考量两种关键的任务间关系：任务干扰（同一参数上多任务梯度更新方向冲突）和任务失衡（各任务学习难度差异导致的优化不均衡）。为此，我们提出任务自适应Transformer（TAT），该框架通过两项创新实现动态任务适配：首先引入任务自适应权重生成策略，通过为各任务生成专属权重参数，消除共享参数上的梯度冲突；其次设计任务自适应损失平衡策略，根据任务学习难度动态调整损失权重，防止任务主导或训练不足。大量实验表明，TAT在PET合成、CT去噪和MRI超分辨率三类MedIR任务中，无论是单任务还是一体化设置均达到最先进性能。代码已开源：https://github.com/Yaziwel/TAT。

English

Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at https://github.com/Yaziwel/TAT.