LayerRecall: 面向视频生成长程一致性的状态条件记忆路由器

摘要

自回归视频扩散通过从有界近期上下文生成片段，实现了可扩展的长视频生成。虽然基于近因的缓存保留了局部连续性，但它会驱逐在主体、物体、场景或属性重新出现时所需的历史线索。现有的记忆机制使模型暴露于非局部历史，但仅能访问并不确保有效使用。我们的分析揭示，视频DiT层对当前、近期和远期上下文表现出不同的偏好，这表明长程记忆需要同时决定检索什么以及在哪里使用它。我们提出了LayerRecall，一种当前条件化的、层选择性的记忆路由器，它检索相关历史K/V状态，并仅将其注入骨干网络特定的记忆敏感层，同时在别处保留局部注意力。为了减少对稀缺的高质量长视野视频和显式记忆分配标签的依赖，我们进一步提出了跨视野预测匹配（CHPM），它使用特权长上下文参考在预测空间监督有界记忆路由器。在100个多镜头评估提示中，LayerRecall在MemoBench和MovieBench上取得了最佳总体结果，同时在其骨干网络上与VBench-Long表现相当，展示了更强的长程恢复能力而无需牺牲局部连续性。定性分析进一步揭示了记忆引导的自校正，即最初不匹配的局部属性恢复到其历史外观，而无需重置进行中的运动或场景结构。额外的分析显示了跨骨干网络的可移植性和可忽略的推理开销。

English

Autoregressive video diffusion enables scalable long-video generation by producing chunks from a bounded recent context. While recency-based caching preserves local continuity, it evicts historical cues needed when subjects, objects, scenes, or attributes reappear. Existing memory mechanisms expose models to nonlocal history, but access alone does not ensure effective use. Our analysis reveals that video DiT layers exhibit distinct preferences for current, recent, and distant context, suggesting that long-range memory requires deciding both what to retrieve and where to use it. We introduce LayerRecall, a current-conditioned, layer-selective memory router that retrieves relevant historical K/V states and injects them only into backbone-specific memory-sensitive layers while preserving local attention elsewhere. To reduce reliance on scarce high-quality long-horizon videos and explicit memory-allocation labels, we further propose Cross-Horizon Prediction Matching (CHPM), which uses a privileged long-context reference to supervise the bounded-memory router in prediction space. Across 100 multi-shot evaluation prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity. Qualitative analyses further reveal memory-guided self-correction, whereby initially mismatched local attributes return to their historical appearance without resetting ongoing motion or scene structure. Additional analyses show cross-backbone portability and negligible inference overhead.