SmartDirector：關鍵幀條件下的電影化視頻生成與敘事節奏控制

摘要

视频的叙事质量从根本上决定了其感知价值。虽然现有视频生成方法能产生视觉上吸引人的内容，但它们主要依赖稀疏的条件信号，如文本提示或首/末帧，这限制了对叙事结构和时间节奏的精确控制。本文提出SmartDirector框架，通过多个关键帧增强视频生成模型的叙事能力。SmartDirector支持灵活生成场景，包括单镜头生成、多镜头叙事合成以及视频扩展。该框架分两个阶段运行：Director-Gen根据提供的关键帧生成低分辨率视频，而Director-SR则利用高分辨率关键帧作为语义锚点来恢复精细细节，从而优化输出。为实现稳健的多关键帧训练，我们构建了一个数据管道，从电影中提取单镜头和多镜头序列。大量实验表明，SmartDirector显著优于现有最先进方法。我们将开源代码以促进进一步研究。

English

The narrative quality of a video fundamentally determines its perceptual value. Although existing video generation methods can produce visually appealing content, they predominantly rely on sparse conditioning signals such as text prompts or first/last frames, which limits precise control over narrative structure and temporal pacing. In this paper, we propose SmartDirector, a framework that enhances the narrative capacity of video generation models through multiple keyframes. SmartDirector supports flexible generation scenarios including single-shot generation, multi-shot narrative synthesis, and video extension. The framework operates in two stages: Director-Gen generates a low-resolution video conditioned on the provided keyframes, and Director-SR refines the output by exploiting high-resolution keyframes as semantic anchors to recover fine-grained details. To enable robust multi-keyframe training, we construct a data pipeline that curates single-shot and multi-shot sequences from movies. Extensive experiments demonstrate that SmartDirector substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research.