Framer：互動式幀插補

Framer: Interactive Frame Interpolation

October 24, 2024

作者: Wen Wang, Qiuyu Wang, Kecheng Zheng, Hao Ouyang, Zhekai Chen, Biao Gong, Hao Chen, Yujun Shen, Chunhua Shen

cs.AI

摘要

我們提出了一種名為 Framer 的互動式幀插補方法，旨在根據使用者的創意產生兩幅圖像之間平滑過渡的幀。具體而言，除了將起始幀和結束幀作為輸入外，我們的方法還支持通過調整一些選定關鍵點的軌跡來自定義過渡過程。這種設計有兩個明顯的好處。首先，融入人類互動有助於緩解由於將一幅圖像轉換為另一幅圖像的眾多可能性而產生的問題，進而實現對局部運動的更精細控制。其次，作為互動的最基本形式，關鍵點有助於建立幀間的對應，增強模型處理具有挑戰性情況的能力（例如，起始幀和結束幀上的物體形狀和風格不同）。值得注意的是，我們的系統還提供了“自動駕駛”模式，其中我們引入了一個模組來自動估計關鍵點並優化軌跡，以簡化實際應用中的使用。大量實驗結果展示了 Framer 在各種應用中的出色性能，例如圖像變形、延時視頻生成、卡通插補等。代碼、模型和界面將被釋出以促進進一步的研究。

English

We propose Framer for interactive frame interpolation, which targets producing smoothly transitioning frames between two images as per user creativity. Concretely, besides taking the start and end frames as inputs, our approach supports customizing the transition process by tailoring the trajectory of some selected keypoints. Such a design enjoys two clear benefits. First, incorporating human interaction mitigates the issue arising from numerous possibilities of transforming one image to another, and in turn enables finer control of local motions. Second, as the most basic form of interaction, keypoints help establish the correspondence across frames, enhancing the model to handle challenging cases (e.g., objects on the start and end frames are of different shapes and styles). It is noteworthy that our system also offers an "autopilot" mode, where we introduce a module to estimate the keypoints and refine the trajectory automatically, to simplify the usage in practice. Extensive experimental results demonstrate the appealing performance of Framer on various applications, such as image morphing, time-lapse video generation, cartoon interpolation, etc. The code, the model, and the interface will be released to facilitate further research.