UniMoMo：基于专家合并的MoE加速方法在大型推荐模型中的应用

摘要

稀疏混合专家（MoE）层通过条件计算扩展推荐能力，然而训练好的检查点仍然在其完整专家库上进行存储和路由。我们研究一个部署问题：在明确的专家预算下，将该检查点转换为更小的标准MoE，且不添加压缩专用的在线模块。为此，我们提出UniMoMo，一个形式化为约束图粗化问题的训练后压缩框架。UniMoMo不依赖参数距离，而是基于功能相似性对专家进行分组，使用无标签校准集来度量专家对共享推荐状态的响应相似程度。为防止性能下降，我们引入一种层自适应保护机制，根据路由暴露度限制高流量专家的合并。在Amazon Beauty、KuaiRec和TenRec上，使用含有2、4和6个MoE块的模型，最终的四专家检查点获得相对于源模型的五次运行平均NDCG@10比率为99.92%--102.30%，实测A100加速比为1.28倍--1.63倍。一种激进的二专家、top-1运行点获得98.36%--104.24%的比率和1.47倍--2.21倍的加速比。这些最终结果评估了完整的转换与适配工作流，并表明训练好的推荐MoE可以在多种服务预算下导出。

English

Sparse mixture-of-experts (MoE) layers expand recommendation capacity through conditional computation, yet a trained checkpoint still stores and routes over its full expert bank. We study a deployment problem: convert that checkpoint to a smaller standard MoE under an explicit expert budget, without adding a compression-specific online module. To address this, we introduce UniMoMo, a post-training compression framework formulated as a constrained graph coarsening problem. Rather than relying on parameter distance, UniMoMo groups experts based on their functional similarity, using an unlabeled calibration set to measure how similarly experts respond to shared recommendation states. To prevent performance degradation, we introduce a layer-adaptive protection mechanism that restricts the merging of high-traffic experts based on their routing exposure. Across Amazon Beauty, KuaiRec, and TenRec with 2, 4, and 6 MoE blocks, the final four-expert checkpoints obtain source-relative five-run mean NDCG@10 ratios of 99.92%--102.30% and measured A100 speedups of 1.28times--1.63times. An aggressive two-expert, top-1 operating point obtains ratios of 98.36%--104.24% and speedups of 1.47times--2.21times. These endpoint results evaluate the complete conversion-and-adaptation workflow and show that a trained recommendation MoE can be exported at multiple serving budgets.