Rep-MTL：释放表征级任务显著性在多任务学习中的潜力

Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning

July 28, 2025

作者: Zedong Wang, Siyuan Li, Dan Xu

cs.AI

摘要

尽管多任务学习在利用跨任务互补知识方面展现出潜力，现有多任务优化（MTO）技术仍局限于通过以优化器为中心的损失缩放和梯度操控策略来解决冲突，却未能带来一致的性能提升。本文主张，在任务交互自然发生的共享表示空间中，蕴含着丰富的信息和操作潜力，这些操作与现有优化器形成互补，尤其是在促进任务间互补性方面，而这一点在MTO中鲜有探索。这一洞见催生了Rep-MTL，它利用表示层面的任务显著性来量化任务特定优化与共享表示学习之间的交互。通过基于熵的惩罚机制和样本级跨任务对齐来引导这些显著性，Rep-MTL旨在通过维持各任务的有效训练而非单纯解决冲突来减轻负迁移，同时明确促进互补信息共享。实验在涵盖任务迁移和领域迁移场景的四个具有挑战性的MTL基准上进行。结果表明，即使搭配基本的等权重策略，Rep-MTL也能实现具有竞争力的性能提升，且效率优异。除标准性能指标外，幂律指数分析进一步证实了Rep-MTL在平衡任务特定学习与跨任务共享方面的有效性。项目页面可访问此处。

English

Despite the promise of Multi-Task Learning in leveraging complementary knowledge across tasks, existing multi-task optimization (MTO) techniques remain fixated on resolving conflicts via optimizer-centric loss scaling and gradient manipulation strategies, yet fail to deliver consistent gains. In this paper, we argue that the shared representation space, where task interactions naturally occur, offers rich information and potential for operations complementary to existing optimizers, especially for facilitating the inter-task complementarity, which is rarely explored in MTO. This intuition leads to Rep-MTL, which exploits the representation-level task saliency to quantify interactions between task-specific optimization and shared representation learning. By steering these saliencies through entropy-based penalization and sample-wise cross-task alignment, Rep-MTL aims to mitigate negative transfer by maintaining the effective training of individual tasks instead pure conflict-solving, while explicitly promoting complementary information sharing. Experiments are conducted on four challenging MTL benchmarks covering both task-shift and domain-shift scenarios. The results show that Rep-MTL, even paired with the basic equal weighting policy, achieves competitive performance gains with favorable efficiency. Beyond standard performance metrics, Power Law exponent analysis demonstrates Rep-MTL's efficacy in balancing task-specific learning and cross-task sharing. The project page is available at HERE.