DistillAlign：協調自迴歸視頻蒸餾中的模式覆蓋與模式尋求

摘要

现有的自回归视频蒸馏方法通常采用基于分布匹配蒸馏（DMD）的多阶段流水线。然而，这些方法通常将初始化阶段与DMD阶段解耦——两者追求不同的目标分布——并主要依据VBench等视觉评分来评判中间学生模型。本文从分布视角重新审视这一设计。鉴于分布匹配损失具有模式寻求特性，良好的初始化应匹配目标DMD教师模型的模式覆盖范围，而非仅仅追求高质量。为分析此问题，我们提出一种分布评估协议，在共享潜空间中衡量学生与教师分布之间的精确度和覆盖度。该协议揭示了视觉评分所隐藏的差异：某些初始化达到高精确度但覆盖度低，导致优化效果欠佳；而覆盖模式的初始化则保留更广的支持范围。此外，即使目标分布已对齐，DMD的反向KL目标在训练后期仍会驱动学生模型趋向教师模型的高概率区域，从而降低覆盖度和多样性。为解决此问题，我们提出联合蒸馏方法，将DMD的模式寻求目标与基于一致性蒸馏的模式覆盖约束相结合。实验表明，我们的方法提升了生成质量、覆盖度和多样性；值得注意的是，即使使用Wan-1.3B DMD教师模型，我们的方法仍优于使用Wan-14B优化的基线，凸显了分布对齐在自回归视频蒸馏中的重要性。

English

Existing autoregressive video distillation methods commonly adopt a Distribution Matching Distillation (DMD)-based multi-stage pipeline. However, they typically decouple the initialization and DMD stages -- which then pursue different target distributions -- and judge the intermediate student mainly by visual scores such as VBench. In this paper, we revisit this design from a distributional perspective. Given the mode-seeking nature of the distribution matching loss, a good initialization should match the mode coverage of the target DMD teacher, rather than merely pursuing high quality. To analyze this, we introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. It exposes differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support. Furthermore, even when the target distributions are aligned, DMD's reverse-KL objective can still drive the student toward high-probability teacher regions in late training, reducing coverage and diversity. To address this, we propose joint distillation, which combines DMD's mode-seeking objective with a Consistency Distillation-based mode-covering constraint. Experiments show that our method improves generation quality, coverage, and diversity; notably, even with a Wan-1.3B DMD teacher, it outperforms baselines refined with Wan-14B, underscoring the importance of distributional alignment in autoregressive video distillation.