用於影片物件插入與層分解的顯式層建模

摘要

大多數影片編輯系統仍缺乏明確的圖層式影片表徵，限制了其在真實合成、物件重複使用與一致化操作上的能力。此限制在影片物件插入與影片圖層分解中尤為明顯，現有方法因缺乏明確的前景圖層監督，只能依賴隱式推論或逐場景最佳化來處理。我們提出 TriLayer 資料集，這是一個大規模的三元組影片資料集，包含對齊的合成影片、背景影片與前景影片，其中前景圖層涵蓋物件外觀及其相關視覺效果。這種明確的監督使模型能直接學習圖層式影片表徵，而非透過隱式推論來取得。基於此資料集，我們提出 DBL-Diffusion，這是一個雙分支擴散框架，透過共享去噪與跨分支交互，共同建模 RGB 合成影像與 RGBA 前景圖層。我們將此框架實例化為兩個任務：DBL-Insert 執行圖層式物件插入，生成明確的 RGBA 圖層以實現真實合成與靈活的後製編輯；DBL-Decompose 執行影片圖層分解，利用三元組監督還原前景與背景圖層。實驗結果顯示，明確的圖層建模能顯著提升插入的真實度與分解的品質。

English

Most video editing systems still lack explicit layered video representations, limiting their ability to perform realistic compositing, object reuse, and consistent manipulation. This limitation is especially pronounced in video object insertion and video layer decomposition, where existing methods rely on implicit inference or per-scene optimization due to the absence of explicit foreground-layer supervision. We introduce TriLayer, a large-scale triplet video dataset containing aligned composite, background, and foreground videos, where the foreground layers include both object appearance and associated visual effects. This explicit supervision enables models to learn layered video representations directly rather than inferring them implicitly. Building on this dataset, we propose DBL-Diffusion, a dual-branch diffusion framework that jointly models RGB composites and RGBA foreground layers through shared denoising and cross-branch interaction. We instantiate the framework in two tasks: DBL-Insert for layered object insertion, which generates explicit RGBA layers for realistic compositing and flexible post-editing, and DBL-Decompose for video layer decomposition, which recovers foreground and background layers using triplet supervision. Experiments demonstrate that explicit layer modeling substantially improves both insertion fidelity and decomposition quality.