TAT：适用于多合一医学图像复原的任务自适应Transformer

TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration

December 16, 2025

作者: Zhiwen Yang, Jiaju Zhang, Yang Yi, Jian Liang, Bingzheng Wei, Yan Xu

cs.AI

摘要

医学图像恢复（MedIR）旨在从低质量医学图像中重建高质量图像。近年来MedIR领域的研究重点已转向能够同时处理多种不同恢复任务的全能模型。然而，由于模态类型和退化类型存在显著差异，使用共享模型处理这些多样化任务时需重点考量两种关键的任务间关系：任务干扰（当不同任务对同一参数产生冲突的梯度更新方向时发生）和任务失衡（由各任务固有学习难度差异导致的优化不均衡）。为应对这些挑战，我们提出任务自适应Transformer（TAT）框架，该创新方案通过两项核心机制实现动态任务适配：首先引入任务自适应权重生成策略，通过为每个任务生成专属权重参数，消除共享权重参数上的潜在梯度冲突；其次采用任务自适应损失平衡策略，根据任务特定学习难度动态调整损失权重，防止任务主导或训练不足。大量实验表明，我们提出的TAT在PET合成、CT去噪和MRI超分辨率三项MedIR任务中，无论是针对单一任务还是全能模型设置，均实现了最先进的性能。代码已开源：https://github.com/Yaziwel/TAT。

English

Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at https://github.com/Yaziwel/TAT.