上下文匹配蒸馏：面向自回归视频蒸馏的教师因果性

摘要

交互式自回归视频生成既需要低延迟的生成过程，又需要精确的在线控制。少步蒸馏通过减少去噪步骤来加速生成，而在线控制则引入了因果约束：帧和块应依赖于历史信息以及生成时可获得的控制信号。然而，现有的视频分布匹配蒸馏（DMD）流程通常使用对完整片段评分的双向教师网络来监督因果少步学生模型。因此，某个目标的评分可能依赖于学生生成该目标时无法获得的未来帧和控制信号，从而使教师监督与学生模型的因果信息集产生错位。我们提出了上下文匹配蒸馏（CMD），这是一种因果 DMD 框架，它将教师监督与每个目标生成时可用的信息对齐。CMD 使用因果教师模型替代双向全片段评分：该教师模型在评估每个目标时无法访问未来帧或控制信号。同一个因果教师模型用于初始化少步学生模型，从而在教师训练、学生蒸馏和推理过程中建立一致的因果表述。除对齐时间信息边界之外，前缀评分（Prefix Scoring）通过在每个目标所对应的、已缓存的学生生成前缀下评估该目标，使监督与学生实际的生成上下文相匹配。前缀扰动（Prefix Corruption）通过扰动训练早期产生的不可靠前缀来进一步稳定训练，同时保持这种目标-上下文对齐。凭借简单的因果设定，CMD 自然可扩展到逐帧和逐块生成、长视频蒸馏以及相机条件蒸馏。实验表明，在短视频和长视频基准上，CMD 在自回归方法中取得了最先进的整体性能，并且显著提升了对时变相机控制的遵循程度。

English

Interactive autoregressive video generation demands both low-latency rollouts and precise online control. Few-step distillation accelerates generation by reducing denoising steps, while online control imposes a causal constraint: frames and blocks should depend on history and controls available during generation. Existing video distribution matching distillation (DMD) pipelines, however, often supervise causal few-step students using bidirectional teachers that score complete clips. The score for a target can therefore depend on future frames and controls that were unavailable when the student generated it, misaligning teacher supervision with the student's causal information set. We introduce Context-Matched Distillation (CMD), a causal DMD framework that aligns teacher supervision with the information available when each target is generated. CMD replaces bidirectional full-clip scoring with a causal teacher that evaluates each target without access to future frames or controls. The same causal teacher initializes the few-step student, establishing a consistent causal formulation across teacher training, student distillation, and inference. Beyond aligning the temporal information boundary, Prefix Scoring matches supervision to the student's realized rollout context by evaluating each target under the cached student-generated prefix that produced it. Prefix Corruption further stabilizes training by perturbing unreliable prefixes produced early in training while preserving this target-context alignment. With a simple causal formulation, CMD naturally extends to frame-wise and chunk-wise generation, long video distillation, and camera-conditioned distillation. Experiments demonstrate state-of-the-art aggregate performance among autoregressive methods on both short- and long-video benchmarks, together with substantially improved adherence to time-varying camera controls.