生成图像动态

摘要

我们提出了一种对场景动态建模图像空间先验的方法。我们的先验是从包含自然振荡运动的真实视频序列中提取的运动轨迹集合中学习的，这些运动包括树木、花朵、蜡烛和风中飘动的衣物。给定一幅单独的图像，我们训练的模型使用频率协调扩散采样过程来预测傅立叶域中每个像素的长期运动表示，我们称之为神经随机运动纹理。这种表示可以转换为跨越整个视频的密集运动轨迹。结合基于图像的渲染模块，这些轨迹可用于许多下游应用，例如将静止图像转换为无缝循环的动态视频，或允许用户在真实图片中与对象进行逼真互动。

English

We present an approach to modeling an image-space prior on scene dynamics. Our prior is learned from a collection of motion trajectories extracted from real video sequences containing natural, oscillating motion such as trees, flowers, candles, and clothes blowing in the wind. Given a single image, our trained model uses a frequency-coordinated diffusion sampling process to predict a per-pixel long-term motion representation in the Fourier domain, which we call a neural stochastic motion texture. This representation can be converted into dense motion trajectories that span an entire video. Along with an image-based rendering module, these trajectories can be used for a number of downstream applications, such as turning still images into seamlessly looping dynamic videos, or allowing users to realistically interact with objects in real pictures.

生成图像动态

Generative Image Dynamics

摘要

Support