ShortOPD：通过从短到长的在线策略蒸馏恢复剪枝后的大语言模型

摘要

结构化剪枝是一种硬件友好的大语言模型压缩方法，但现有验证多集中于多项选择识别任务，而实际部署所需的自由文本生成中，相同压缩检查点可能完全失效。两种观察揭示了这一差距：首先，贪心式pass@1在压缩后几乎消失，但重复采样下pass@k显著恢复——有用生成被降级而非消除。其次，可恢复区域的主要失效模式是后缀重复。因此，恢复训练应基于压缩模型自身的在策略状态，辅以密集的令牌级监督——在策略蒸馏（OPD）通过复用压缩前模型作为冻结教师来实现这一点。然而，长序列在策略展开会将早期恢复预算浪费在低信息量的重复后缀上，延缓损失下降。为缓解此浪费，我们提出\shortopd——一种从短到长的OPD调度策略：检测经教师确认的重复后缀，将存活前缀作为每次展开的有效长度，并将未来展开预算分配给当前策略可实际使用的有效长度。在数学、代码和开放式生成任务中，\shortopd将压缩模型得分提升至未恢复值的约9倍，为标准恢复方案（无知识蒸馏的SFT、KD、SeqKD）的1.6至4.4倍；且以更少的训练时间（8.5小时 vs 35.9小时）和减少71%的展开令牌数，在固定8192令牌展开长度下实现两点内的性能匹配。我们希望这一方案能推动结构化剪枝走出仅依赖困惑度和多项选择基准的边际增益，向部署级生成质量迈进一步。

English

Structured pruning is a hardware-friendly way to compress LLMs, but it is mostly validated on multiple-choice recognition tasks, while the same compressed checkpoints can collapse on the free-form generation that deployment actually requires. Two observations trace this gap. First, greedy pass@1 nearly vanishes after compression, yet pass@k recovers substantially under repeated sampling: useful generations are demoted, not erased. Second, the recoverable regime fails mainly through suffix repetition. Recovery should therefore train on the compressed model's own on-policy states with dense token-level supervision, which On-Policy Distillation (OPD) provides by reusing the pre-compression model as a frozen teacher. However, long on-policy rollouts spend early recovery budget on low-information repetitive suffixes, delaying loss descent. To mitigate this waste, we propose \shortopd, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes, treats the surviving prefix as each rollout's effective length, and allocates future rollout budgets to the effective lengths the policy can currently use. Across math, code, and open-ended generation, \shortopd\ raises the compressed model's score to about 9times its unrecovered value and 1.6--4.4times standard recovery recipes (SFT w/o KD, KD, and SeqKD), and it matches a fixed 8192-token rollout horizon within two points using a quarter of the training time (8.5 vs.\ 35.9 hours) and 71% fewer rollout tokens. We hope this recipe helps move structured pruning beyond marginal gains on perplexity and multiple-choice benchmarks, a step closer to deployment-ready generation quality.