生成影像動態

摘要

我們提出了一種對場景動態建模的影像空間先驗方法。我們的先驗是從包含自然振盪運動的真實視頻序列中提取的運動軌跡集合中學習的，這些運動包括樹木、花朵、蠟燭和風中飄動的衣物等。給定一張單獨的圖像，我們訓練的模型使用頻率協調擴散採樣過程，在傅立葉域中預測每個像素的長期運動表示，我們稱之為神經隨機運動紋理。這種表示可以轉換為涵蓋整個視頻的密集運動軌跡。連同基於圖像的渲染模塊，這些軌跡可以用於多個下游應用，例如將靜止圖像轉換為無縫循環動態視頻，或讓用戶與真實圖片中的物體進行逼真互動。

English

We present an approach to modeling an image-space prior on scene dynamics. Our prior is learned from a collection of motion trajectories extracted from real video sequences containing natural, oscillating motion such as trees, flowers, candles, and clothes blowing in the wind. Given a single image, our trained model uses a frequency-coordinated diffusion sampling process to predict a per-pixel long-term motion representation in the Fourier domain, which we call a neural stochastic motion texture. This representation can be converted into dense motion trajectories that span an entire video. Along with an image-based rendering module, these trajectories can be used for a number of downstream applications, such as turning still images into seamlessly looping dynamic videos, or allowing users to realistically interact with objects in real pictures.

生成影像動態

Generative Image Dynamics

摘要

Support