LiveEdit：邁向即時基於擴散模型的串流影片編輯

摘要

串流影片編輯已取得快速進展，但實際部署仍受限於兩大核心問題：維持背景與未編輯區域隨時間的穩定性，以及達成即時互動場景所需的低延遲。與此同時，近年串流影片生成方法大多針對合成任務開發，因嚴格要求內容保存與區域特定控制而無法直接應用於編輯。本研究中，我們提出新穎的串流影片編輯框架，以因果方式逐幀進行編輯，兼具強大的內容保存與即時響應能力。核心設計為三階段蒸餾流程，逐步將強大的雙向基礎模型之編輯能力轉移至高效的單向串流編輯器，在維持視覺逼真度的同時實現穩定的長期編輯。為進一步支援即時部署，我們引入針對擴增實境設計的遮罩快取機制，跨幀重複使用區域相關計算，大幅減少冗餘處理並加速推論。最後，我們建立專屬的串流影片編輯基準測試。廣泛評估顯示，本方法在串流基準中達到最先進的視覺品質，同時將推論速度大幅提升至每秒12.66幀，適合應用於互動式與擴增實境場景。

English

Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.