聚焦关键：利用显著性引导的扩散MoE精确路由

摘要

混合专家（MoE）架构已成为扩展视觉生成扩散模型的强大范式。近年来的研究主要聚焦于跨多样令牌自适应分配计算资源，以提升效率与性能。然而，我们发现现有扩散MoE框架存在路由分配问题：路由无法准确为显著令牌分配更多计算资源。我们的分析将此归因于路由在去噪过程中依赖于受噪声污染的潜在特征。此类随机噪声掩盖了关键的结构与纹理信息，从而阻碍路由有效区分显著令牌。为解决此问题，我们提出SharpMoE——一种基于后训练框架的显著性感知精准路由机制，该机制利用清洁潜在特征作为无噪声的引导信号进行路由。通过绕过噪声失真的输入，SharpMoE为路由提供清晰的显著性引导，使其即使在高度噪声阶段也能识别显著令牌。此外，我们引入轨迹路由损失来约束多步去噪轨迹中的计算分配，确保生成过程沿展开路径的精准资源分配。大量实验表明，SharpMoE作为一种通用即插即用方案，可进一步增强预训练且已收敛的MoE模型，在视觉生成任务中实现最先进性能。

English

Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.