TVG：一种基于扩散模型的无需训练的过渡视频生成方法

TVG: A Training-free Transition Video Generation Method with Diffusion Models

August 24, 2024

作者: Rui Zhang, Yaosen Chen, Yuegen Liu, Wei Wang, Xuming Wen, Hongxia Wang

cs.AI

摘要

过渡视频在媒体制作中发挥着至关重要的作用，增强了视觉叙事的流畅性和连贯性。传统方法如变形通常缺乏艺术吸引力，需要专业技能，限制了它们的有效性。基于扩散模型的视频生成的最新进展为创建过渡提供了新的可能性，但面临诸如帧间关系建模不足和内容突变等挑战。我们提出了一种新颖的无需训练的过渡视频生成（TVG）方法，使用视频级扩散模型来解决这些限制，无需额外训练。我们的方法利用高斯过程回归（GPR）来建模潜在表示，确保帧间过渡平滑而动态。此外，我们引入基于插值的条件控制和频率感知的双向融合（FBiF）架构，以增强时间控制和过渡可靠性。对基准数据集和自定义图像对的评估表明，我们的方法在生成高质量平滑过渡视频方面的有效性。代码提供在https://sobeymil.github.io/tvg.com。

English

Transition videos play a crucial role in media production, enhancing the flow and coherence of visual narratives. Traditional methods like morphing often lack artistic appeal and require specialized skills, limiting their effectiveness. Recent advances in diffusion model-based video generation offer new possibilities for creating transitions but face challenges such as poor inter-frame relationship modeling and abrupt content changes. We propose a novel training-free Transition Video Generation (TVG) approach using video-level diffusion models that addresses these limitations without additional training. Our method leverages Gaussian Process Regression (GPR) to model latent representations, ensuring smooth and dynamic transitions between frames. Additionally, we introduce interpolation-based conditional controls and a Frequency-aware Bidirectional Fusion (FBiF) architecture to enhance temporal control and transition reliability. Evaluations of benchmark datasets and custom image pairs demonstrate the effectiveness of our approach in generating high-quality smooth transition videos. The code are provided in https://sobeymil.github.io/tvg.com.