TVG：一種基於擴散模型的無需訓練的過渡影片生成方法

TVG: A Training-free Transition Video Generation Method with Diffusion Models

August 24, 2024

作者: Rui Zhang, Yaosen Chen, Yuegen Liu, Wei Wang, Xuming Wen, Hongxia Wang

cs.AI

摘要

過渡影片在媒體製作中扮演著關鍵角色，提升視覺敘事的流暢性和連貫性。傳統方法如形變在藝術吸引力上常有所欠缺，並需要專業技能，限制了其效果。最近基於擴散模型的影片生成技術帶來了新的可能性，但面臨著像是模型化幀間關係不足和內容突變等挑戰。我們提出了一種新穎的無需訓練的過渡影片生成（TVG）方法，採用影片級擴散模型來解決這些限制，無需額外訓練。我們的方法利用高斯過程回歸（GPR）來建模潛在表示，確保幀間過渡平滑且動態。此外，我們引入基於插值的條件控制和頻率感知的雙向融合（FBiF）架構，以增強時間控制和過渡可靠性。對基準數據集和自定義圖像對的評估顯示了我們方法在生成高質量平滑過渡影片方面的有效性。程式碼可在 https://sobeymil.github.io/tvg.com 中找到。

English

Transition videos play a crucial role in media production, enhancing the flow and coherence of visual narratives. Traditional methods like morphing often lack artistic appeal and require specialized skills, limiting their effectiveness. Recent advances in diffusion model-based video generation offer new possibilities for creating transitions but face challenges such as poor inter-frame relationship modeling and abrupt content changes. We propose a novel training-free Transition Video Generation (TVG) approach using video-level diffusion models that addresses these limitations without additional training. Our method leverages Gaussian Process Regression (GPR) to model latent representations, ensuring smooth and dynamic transitions between frames. Additionally, we introduce interpolation-based conditional controls and a Frequency-aware Bidirectional Fusion (FBiF) architecture to enhance temporal control and transition reliability. Evaluations of benchmark datasets and custom image pairs demonstrate the effectiveness of our approach in generating high-quality smooth transition videos. The code are provided in https://sobeymil.github.io/tvg.com.