LiveEdit: 迈向基于扩散模型的实时流式视频编辑

摘要

流媒体视频编辑技术虽已取得快速进展，但其实际部署仍受两大核心问题制约：如何长期维持稳定背景与非编辑区域的连贯性，以及如何实现实时交互场景所需的低延迟。与此同时，现有流媒体视频生成方法多针对合成任务开发，因需严格保留原内容并进行区域特异性控制，难以直接应用于编辑场景。本文提出一种全新的流媒体视频编辑框架，以因果、逐帧编辑方式实现强内容保持与实时响应。其核心设计是一种三阶段蒸馏流程，将强大的双向基础模型编辑能力逐步迁移至高效单向流媒体编辑器，在确保视觉保真度的前提下实现稳定长程编辑。为支撑实时部署，我们进一步引入面向增强现实的掩膜缓存机制，跨帧复用区域相关计算，显著减少冗余处理并加速推理。最后，我们为流媒体视频编辑建立了专用基准测试集。大量评估表明，本方法在流媒体基线中实现最优视觉质量，同时将推理速度大幅提升至12.66帧/秒，适用于交互式与增强现实应用场景。

English

Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.