UniMoMo：基於專家合併的大型推薦模型MoE加速方法

摘要

稀疏混合專家（MoE）層透過條件計算擴展了推薦系統的容量，然而訓練完成的檢查點仍會在其完整的專家庫上進行儲存與路由。我們研究一個部署問題：在明確的專家預算下，將該檢查點轉換為較小的標準MoE，且不添加任何壓縮專用的線上模組。為了解決此問題，我們提出UniMoMo，這是一個以約束圖粗化問題形式建構的訓練後壓縮框架。有別於依賴參數距離，UniMoMo根據專家的功能相似性進行分組，使用未標註的校準集來衡量專家對共享推薦狀態的反應相似程度。為防止效能退化，我們引入了一種層自適應保護機制，根據路由曝光度限制高流量專家的合併。在Amazon Beauty、KuaiRec和TenRec上，分別使用2、4和6個MoE區塊，最終的四專家檢查點獲得了相對於原始模型的五次運行平均NDCG@10比率為99.92%–102.30%，以及實測A100加速比為1.28倍–1.63倍。一個激進的兩專家、top-1操作點則獲得了98.36%–104.24%的比率和1.47倍–2.21倍的加速比。這些端點結果評估了完整的轉換與調適流程，顯示訓練完成的推薦MoE可以依照多種服務預算進行匯出。

English

Sparse mixture-of-experts (MoE) layers expand recommendation capacity through conditional computation, yet a trained checkpoint still stores and routes over its full expert bank. We study a deployment problem: convert that checkpoint to a smaller standard MoE under an explicit expert budget, without adding a compression-specific online module. To address this, we introduce UniMoMo, a post-training compression framework formulated as a constrained graph coarsening problem. Rather than relying on parameter distance, UniMoMo groups experts based on their functional similarity, using an unlabeled calibration set to measure how similarly experts respond to shared recommendation states. To prevent performance degradation, we introduce a layer-adaptive protection mechanism that restricts the merging of high-traffic experts based on their routing exposure. Across Amazon Beauty, KuaiRec, and TenRec with 2, 4, and 6 MoE blocks, the final four-expert checkpoints obtain source-relative five-run mean NDCG@10 ratios of 99.92%--102.30% and measured A100 speedups of 1.28times--1.63times. An aggressive two-expert, top-1 operating point obtains ratios of 98.36%--104.24% and speedups of 1.47times--2.21times. These endpoint results evaluate the complete conversion-and-adaptation workflow and show that a trained recommendation MoE can be exported at multiple serving budgets.