Vera：一種用於內容保留影片編輯的分層式擴散模型

摘要

視頻擴散模型在影片生成與編輯方面已實現顯著進展。然而，內容保留仍是核心挑戰：現有方法會重新生成每一個像素，往往改變本應保持不變的要素，例如角色或背景場景。我們提出Vera，一個用於內容保留影片編輯的分層擴散框架。Vera並非重新生成整個影片，而是生成一個編輯圖層及其對應的Alpha遮罩，用於與原始影片合成，透過設計將創意編輯與內容保留分離開來。為促進與原始影片的一致性合成，我們將文字轉影片DiT擴展為變壓器混合（MoT）架構，每個圖層有各自的DiT，透過聯合自注意力機制進行互動。為支援Vera的訓練，我們進一步建構一個高品質的分層資料集，包含精確的Alpha遮罩、多樣的場景與動態，以及視覺效果。在我們的定量基準與人類偏好研究中，Vera僅使用486K幀的分層訓練資料，即在內容保留方面優於領先的開源影片編輯模型，同時在編輯品質上保持競爭力。

English

Video diffusion models have enabled remarkable progress in video generation and editing. However, content preservation remains a core challenge: existing methods regenerate every pixel and often alter elements that should remain unchanged, such as characters or background scenes. We introduce Vera, a layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design. To encourage coherent composition with the source video, we extend the text-to-video DiT into a Mixture-of-Transformers (MoT) architecture, with separate DiTs for each layer that interact through joint self-attention. To support the training of Vera, we further construct a high-quality layered dataset with accurate alpha mattes, diverse scenes and dynamics, and visual effects. Across our quantitative benchmark and human preference study, Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.