HodgeCover：高階拓撲覆蓋驅動稀疏混合專家模型的壓縮

摘要

稀疏混合專家（MoE）層將令牌路由至少數專家，對此類層進行無學習壓縮可減少推理成本而無需重新訓練。一個微妙障礙阻礙了該系列中的所有現有壓縮器：三個專家可能兩兩相容，但合併時卻形成不可約循環，因此任何基於成對訊號對專家進行排序的分數，在結構上無法感知哪些三元組可共同合併。我們證明此障礙是一個精確的數學對象，即二維複形上單純拉普拉斯算子的諧波核，該複形的頂點為專家、邊承載KL合併障礙、面承載三元組障礙；對邊障礙訊號進行霍奇分解可精確隔離該核。我們將此診斷轉化為選擇目標：霍奇覆蓋貪婪地覆蓋諧波關鍵邊與三元組關鍵三角形，其混合變體則將霍奇覆蓋與現成的權重剪枝結合應用於倖存專家。在三種開放權重稀疏MoE骨幹上，面對激進的專家壓縮，霍奇覆蓋在專家壓縮維度上與最先進的無學習基準相當，在混合維度的激進壓縮前沿領先，並獨特地平衡了所有四個霍奇分量上的保留質量。這些結果表明，揭示學習所得MoE結構的諧波核，會改變在最關鍵場景下取勝的壓縮器。

English

Sparse Mixture-of-Experts (MoE) layers route tokens through a handful of experts, and learning-free compression of these layers reduces inference cost without retraining. A subtle obstruction blocks every existing compressor in this family: three experts can each be pairwise compatible yet form an irreducible cycle when merged together, so any score that ranks experts on pairwise signals is structurally blind to which triples are jointly mergeable. We show the obstruction is a precise mathematical object, the harmonic kernel of the simplicial Laplacian on a 2-complex whose vertices are experts, whose edges carry KL merge barriers, and whose faces carry triplet barriers; Hodge-decomposing the edge-barrier signal isolates the kernel exactly. We turn the diagnostic into a selection objective: HodgeCover greedily covers the harmonic-critical edges and triplet-critical triangles, and a hybrid variant of HodgeCover pairs it with off-the-shelf weight pruning on survivors. On three open-weight Sparse MoE backbones under aggressive expert reduction, HodgeCover matches state-of-the-art learning-free baselines on the expert-reduction axis, leads on the aggressive-compression frontier of the hybrid axis, and uniquely balances retained mass across all four Hodge components. These results show that exposing the harmonic kernel of a learned MoE structure changes which compressor wins at the regime that matters most.