自梯度驅動：原生長視頻外推

摘要

近期，自回归视频扩散方法日益建立在「自强制」（Self Forcing）框架之上——学生模型在其自身展开生成的历史序列而非真实视频上下文中进行训练。这虽然减少了曝光偏差，但历史键值缓存仍仅作为冻结的展开状态供后续帧使用。因此，未来帧的损失无法监督早期生成的潜变量应如何被写入对后续视频潜变量生成更有用的键值与值中，我们将此称为「历史上下文-梯度鸿沟」。为此，我们提出「自梯度强制」（Self Gradient Forcing, SGF），一种无需对整个串行展开过程进行反向传播的双阶段训练策略，用以恢复缺失的监督信号。第一阶段执行无梯度的自回归展开，其过程与推理一致，并在采样的去噪退出步骤中，同时记录模型自生成的上下文与输入的噪声潜变量。第二阶段针对记录的退出步骤执行并行的上下文梯度重建：自生成的上下文作为停止梯度的干净潜变量输入，模型在此阶段重新计算上下文的键值表示以及从未来帧到上下文的因果注意力。由此，SGF能够在原生自回归训练目标中补全缺失的记忆写入监督，利用未来视频潜变量的损失来训练模型将上下文编码为更有效的因果记忆。在不同初始化条件下的大规模长程逐帧与逐块实验中，SGF均实现了比自强制更强的原生长视频外推能力，尤其是在主体一致性、背景/布局稳定性及时间稳定性方面。值得注意的是，仅使用5秒的训练窗口，SGF即可外推生成持续数分钟的视频。代码与模型将公开，以推动自回归视频生成研究的发展。

English

Recent autoregressive video diffusion methods are increasingly built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground-truth video contexts. This reduces exposure bias, but the historical key-value cache is still used by future frames only as frozen rollout state. As a result, future losses cannot supervise how earlier generated latents should be written into more useful keys and values for later video-latent generation. We call this the historical context-gradient gap. We propose Self Gradient Forcing (SGF), a two-pass training strategy that restores this missing supervision signal without backpropagating through the full serial rollout. Pass 1 performs a no-gradient autoregressive rollout matching inference and, at a sampled denoising exit step, records both the self-generated context and the noisy latents fed to the model. Pass 2 performs parallel context-gradient reconstruction for the recorded exit step. The generated context is used as stop-gradient clean-latent input, while the model recomputes the context KV representations and future-to-context causal attention. Thus, SGF provides the missing memory-writing supervision within the native autoregressive training objective, using losses on future video latents to train the model to encode context into more effective causal memory. Across extensive long-horizon frame-wise and chunk-wise experiments under different initializations, SGF achieves stronger native long-video extrapolation than Self Forcing, especially in subject identity, background/layout consistency, and temporal stability. Remarkably, using only a 5-second training window, SGF can extrapolate to videos lasting several minutes. Code and models will be released to advance research on autoregressive video generation.