CCEdit:通过扩散模型实现创意和可控视频编辑
CCEdit: Creative and Controllable Video Editing via Diffusion Models
September 28, 2023
作者: Ruoyu Feng, Wenming Weng, Yanhui Wang, Yuhui Yuan, Jianmin Bao, Chong Luo, Zhibo Chen, Baining Guo
cs.AI
摘要
在这项工作中,我们提出了CCEdit,这是一个多功能框架,旨在解决创意和可控视频编辑的挑战。CCEdit满足了各种用户编辑需求,并通过一种创新方法解耦视频结构和外观,从而实现增强的创意控制。我们利用基础的ControlNet架构来保持结构完整性,同时无缝集成了与文本到图像生成的最新个性化技术兼容的可调节时间模块,如DreamBooth和LoRA。此外,我们引入了基于参考条件的视频编辑,使用户能够通过更易管理的关键帧编辑过程精确控制视频编辑。我们进行了大量实验评估,证实了所提出的CCEdit框架的卓越功能和编辑能力。演示视频可在https://www.youtube.com/watch?v=UQw4jq-igN4 上观看。
English
In this work, we present CCEdit, a versatile framework designed to address
the challenges of creative and controllable video editing. CCEdit accommodates
a wide spectrum of user editing requirements and enables enhanced creative
control through an innovative approach that decouples video structure and
appearance. We leverage the foundational ControlNet architecture to preserve
structural integrity, while seamlessly integrating adaptable temporal modules
compatible with state-of-the-art personalization techniques for text-to-image
generation, such as DreamBooth and LoRA.Furthermore, we introduce
reference-conditioned video editing, empowering users to exercise precise
creative control over video editing through the more manageable process of
editing key frames. Our extensive experimental evaluations confirm the
exceptional functionality and editing capabilities of the proposed CCEdit
framework. Demo video is available at
https://www.youtube.com/watch?v=UQw4jq-igN4.