DistillAlign：协调自回归视频蒸馏中的模式覆盖与模式搜索

摘要

现有自回归视频蒸馏方法普遍采用基于分布匹配蒸馏（DMD）的多阶段流水线。然而，这类方法通常将初始化阶段与DMD阶段解耦——二者追求不同的目标分布——并且主要通过视觉评分（如VBench）来评估中间学生模型的质量。本文从分布视角重新审视这一设计。考虑到分布匹配损失具有模式寻求特性，良好的初始化应匹配目标DMD教师的模式覆盖范围，而非仅追求高视觉质量。为此，我们引入一种分布评估协议，在共享潜在空间中测量学生分布与教师分布之间的精确度与覆盖度。该协议揭示了视觉评分无法体现的差异：部分初始化方案虽能达到高精确度但覆盖度低，导致后续优化效果欠佳；而覆盖模式的初始化方案则能保留更广泛的支持域。此外，即使目标分布一致，DMD的反向KL目标函数在训练后期仍可能驱动学生模型向高概率的教师区域收敛，从而降低覆盖度与多样性。针对这一问题，我们提出联合蒸馏方法，将DMD的模式寻求目标与基于一致性蒸馏的模式覆盖约束相结合。实验表明，本文方法在生成质量、覆盖度和多样性上均有提升；值得注意的是，即使采用Wan-1.3B DMD教师，其性能仍优于使用Wan-14B教师精调的基线模型，凸显了自回归视频蒸馏中分布对齐的重要性。

English

Existing autoregressive video distillation methods commonly adopt a Distribution Matching Distillation (DMD)-based multi-stage pipeline. However, they typically decouple the initialization and DMD stages -- which then pursue different target distributions -- and judge the intermediate student mainly by visual scores such as VBench. In this paper, we revisit this design from a distributional perspective. Given the mode-seeking nature of the distribution matching loss, a good initialization should match the mode coverage of the target DMD teacher, rather than merely pursuing high quality. To analyze this, we introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space. It exposes differences hidden by visual scores: some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support. Furthermore, even when the target distributions are aligned, DMD's reverse-KL objective can still drive the student toward high-probability teacher regions in late training, reducing coverage and diversity. To address this, we propose joint distillation, which combines DMD's mode-seeking objective with a Consistency Distillation-based mode-covering constraint. Experiments show that our method improves generation quality, coverage, and diversity; notably, even with a Wan-1.3B DMD teacher, it outperforms baselines refined with Wan-14B, underscoring the importance of distributional alignment in autoregressive video distillation.