AnyI2V：基于运动控制的任意条件图像动画生成

摘要

近期，视频生成领域，尤其是扩散模型方面的进展，显著推动了文本到视频（T2V）和图像到视频（I2V）合成技术的发展。然而，在有效整合动态运动信号与灵活空间约束方面仍存在挑战。现有的T2V方法通常依赖文本提示，这本质上难以精确控制生成内容的空间布局。相比之下，I2V方法受限于对真实图像的依赖，限制了合成内容的可编辑性。尽管部分方法通过引入ControlNet实现了基于图像的条件控制，但它们往往缺乏明确的运动控制，且训练过程计算成本高昂。为克服这些局限，我们提出了AnyI2V，一个无需训练即可根据用户定义的运动轨迹为任意条件图像赋予动画效果的框架。AnyI2V支持更广泛的条件图像模态，包括ControlNet不支持的网格和点云等数据类型，从而实现了更灵活多样的视频生成。此外，它还支持混合条件输入，并通过LoRA和文本提示实现风格迁移与编辑。大量实验证明，所提出的AnyI2V在空间与运动控制的视频生成中表现卓越，为这一领域提供了新的视角。代码可在https://henghuiding.com/AnyI2V/获取。

English

Recent advancements in video generation, particularly in diffusion models, have driven notable progress in text-to-video (T2V) and image-to-video (I2V) synthesis. However, challenges remain in effectively integrating dynamic motion signals and flexible spatial constraints. Existing T2V methods typically rely on text prompts, which inherently lack precise control over the spatial layout of generated content. In contrast, I2V methods are limited by their dependence on real images, which restricts the editability of the synthesized content. Although some methods incorporate ControlNet to introduce image-based conditioning, they often lack explicit motion control and require computationally expensive training. To address these limitations, we propose AnyI2V, a training-free framework that animates any conditional images with user-defined motion trajectories. AnyI2V supports a broader range of modalities as the conditional image, including data types such as meshes and point clouds that are not supported by ControlNet, enabling more flexible and versatile video generation. Additionally, it supports mixed conditional inputs and enables style transfer and editing via LoRA and text prompts. Extensive experiments demonstrate that the proposed AnyI2V achieves superior performance and provides a new perspective in spatial- and motion-controlled video generation. Code is available at https://henghuiding.com/AnyI2V/.

AnyI2V：基于运动控制的任意条件图像动画生成

AnyI2V: Animating Any Conditional Image with Motion Control

摘要

Support