聚焦關鍵：基於顯著性引導的擴散混合專家模型精確路由

摘要

混合專家（MoE）架構已成為視覺生成中擴展擴散模型的強大典範。近期進展專注於自適應地跨不同特徵分配計算資源，以提升效率與效能。然而，我們發現現有擴散MoE框架中存在一個路由分配問題：路由器未能準確地將更多計算資源分配給顯著特徵。我們的分析將此失敗歸因於路由器在整個去噪過程中依賴於受雜訊干擾的潛在特徵。此類隨機雜訊遮蔽了關鍵的結構與紋理資訊，從而阻止路由器有效區分顯著特徵。為解決此問題，我們提出SharpMoE，這是一個具有利用顯著性的精確路由機制的訓練後框架，該機制利用無雜訊的潛在特徵作為路由的無噪音引導訊號。透過繞過受雜訊扭曲的輸入，SharpMoE為路由器提供清晰的顯著性引導，使其即使在高度雜訊階段也能識別顯著特徵。此外，我們引入了一種軌跡路由損失，以約束多步去噪軌跡中的計算分配，確保沿著生成展開過程中的精確資源分配。廣泛的實驗表明，SharpMoE作為一種通用、即插即用的解決方案，可進一步增強預訓練且已收斂的MoE模型，在視覺生成中達到最先進的表現。

English

Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve efficiency and performance. However, we identify a routing assignment problem in existing diffusion MoE frameworks: the router fails to accurately allocate more computational resources to salient tokens. Our analysis attributes this failure to the router's reliance on noise-corrupted latent features throughout the denoising process. Such stochastic noise obscures the critical structural and textural information, thereby preventing the router from effectively distinguishing salient tokens. To address this, we propose SharpMoE, a post-training framework with a saliency-harnessing accurate routing mechanism, which utilizes clean latent features as a noise-free guidance signal for routing. By bypassing the noise-distorted inputs, SharpMoE provides the router with clear saliency guidance, enabling the identification of salient tokens even in high-noise stages. Furthermore, we introduce a trajectory routing loss to constrain the compute allocation throughout the multi-step denoising trajectory, ensuring precise resource allocation along the generation rollout. Extensive experiments demonstrate that SharpMoE serves as a versatile, plug-and-play solution that further enhances the pretrained, converged MoE models, achieving state-of-the-art performance in visual generation.