AnyI2V：基於運動控制實現任意條件圖像的動畫生成

摘要

近期在視頻生成領域，尤其是擴散模型方面的進展，顯著推動了文本到視頻（T2V）和圖像到視頻（I2V）合成的發展。然而，在有效整合動態運動信號和靈活的空間約束方面仍存在挑戰。現有的T2V方法通常依賴於文本提示，這本質上缺乏對生成內容空間佈局的精確控制。相比之下，I2V方法則受限於其對真實圖像的依賴，這限制了合成內容的可編輯性。儘管一些方法引入了ControlNet以實現基於圖像的條件控制，但它們往往缺乏明確的運動控制，並且需要耗費大量計算資源進行訓練。為解決這些限制，我們提出了AnyI2V，這是一個無需訓練的框架，能夠根據用戶定義的運動軌跡為任何條件圖像生成動畫。AnyI2V支持更廣泛的條件圖像模態，包括ControlNet不支持的數據類型，如網格和點雲，從而實現更靈活多樣的視頻生成。此外，它還支持混合條件輸入，並通過LoRA和文本提示實現風格轉換和編輯。大量實驗表明，所提出的AnyI2V在空間和運動控制的視頻生成方面表現卓越，並提供了新的視角。代碼可在https://henghuiding.com/AnyI2V/獲取。

English

Recent advancements in video generation, particularly in diffusion models, have driven notable progress in text-to-video (T2V) and image-to-video (I2V) synthesis. However, challenges remain in effectively integrating dynamic motion signals and flexible spatial constraints. Existing T2V methods typically rely on text prompts, which inherently lack precise control over the spatial layout of generated content. In contrast, I2V methods are limited by their dependence on real images, which restricts the editability of the synthesized content. Although some methods incorporate ControlNet to introduce image-based conditioning, they often lack explicit motion control and require computationally expensive training. To address these limitations, we propose AnyI2V, a training-free framework that animates any conditional images with user-defined motion trajectories. AnyI2V supports a broader range of modalities as the conditional image, including data types such as meshes and point clouds that are not supported by ControlNet, enabling more flexible and versatile video generation. Additionally, it supports mixed conditional inputs and enables style transfer and editing via LoRA and text prompts. Extensive experiments demonstrate that the proposed AnyI2V achieves superior performance and provides a new perspective in spatial- and motion-controlled video generation. Code is available at https://henghuiding.com/AnyI2V/.

AnyI2V：基於運動控制實現任意條件圖像的動畫生成

AnyI2V: Animating Any Conditional Image with Motion Control

摘要

Support