显式层建模用于视频对象插入与层分解

摘要

大多数视频编辑系统仍然缺乏显式的分层视频表示，这限制了它们进行逼真合成、对象复用和一致性操作的能力。这一局限在视频对象插入和视频层分解中尤为突出，现有方法由于缺乏显式的前景层监督，往往依赖隐式推理或逐场景优化。我们提出了TriLayer，一个大规模三元组视频数据集，包含对齐的合成视频、背景视频和前景视频，其中前景层不仅包含对象外观，还包含相关的视觉效果。这种显式监督使得模型能够直接学习分层视频表示，而非隐式推断。基于该数据集，我们提出了DBL-Diffusion，一个双分支扩散框架，通过共享去噪和跨分支交互，联合建模RGB合成图像和RGBA前景层。我们将该框架实例化为两个任务：DBL-Insert用于分层对象插入，生成显式的RGBA层以实现逼真合成和灵活后期编辑；DBL-Decompose用于视频层分解，利用三元组监督恢复前景和背景层。实验表明，显式层建模在插入逼真度和分解质量上均有显著提升。

English

Most video editing systems still lack explicit layered video representations, limiting their ability to perform realistic compositing, object reuse, and consistent manipulation. This limitation is especially pronounced in video object insertion and video layer decomposition, where existing methods rely on implicit inference or per-scene optimization due to the absence of explicit foreground-layer supervision. We introduce TriLayer, a large-scale triplet video dataset containing aligned composite, background, and foreground videos, where the foreground layers include both object appearance and associated visual effects. This explicit supervision enables models to learn layered video representations directly rather than inferring them implicitly. Building on this dataset, we propose DBL-Diffusion, a dual-branch diffusion framework that jointly models RGB composites and RGBA foreground layers through shared denoising and cross-branch interaction. We instantiate the framework in two tasks: DBL-Insert for layered object insertion, which generates explicit RGBA layers for realistic compositing and flexible post-editing, and DBL-Decompose for video layer decomposition, which recovers foreground and background layers using triplet supervision. Experiments demonstrate that explicit layer modeling substantially improves both insertion fidelity and decomposition quality.