LAVE：基于LLM的视频编辑代理辅助和语言增强

LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing

February 15, 2024

作者: Bryan Wang, Yuliang Li, Zhaoyang Lv, Haijun Xia, Yan Xu, Raj Sodhi

cs.AI

摘要

视频制作变得越来越受欢迎，但编辑所需的专业知识和努力常常对初学者构成障碍。在本文中，我们探讨了将大型语言模型（LLMs）整合到视频编辑工作流程中以减少这些障碍。我们的设计愿景体现在LAVE中，这是一个提供LLM动力代理辅助和语言增强编辑功能的新颖系统。LAVE自动生成用户素材的语言描述，为LLM处理视频和协助编辑任务奠定基础。当用户提供编辑目标时，代理规划并执行相关操作以实现目标。此外，LAVE允许用户通过代理或直接UI操作来编辑视频，提供灵活性并实现对代理操作的手动调整。我们的用户研究包括从初学者到熟练编辑人员的八名参与者，证明了LAVE的有效性。结果还揭示了用户对所提出的LLM辅助编辑范式以及其对用户创造力和共同创作感的看法。根据这些发现，我们提出了设计启示，以指导未来代理辅助内容编辑的发展。

English

Video creation has become increasingly popular, yet the expertise and effort required for editing often pose barriers to beginners. In this paper, we explore the integration of large language models (LLMs) into the video editing workflow to reduce these barriers. Our design vision is embodied in LAVE, a novel system that provides LLM-powered agent assistance and language-augmented editing features. LAVE automatically generates language descriptions for the user's footage, serving as the foundation for enabling the LLM to process videos and assist in editing tasks. When the user provides editing objectives, the agent plans and executes relevant actions to fulfill them. Moreover, LAVE allows users to edit videos through either the agent or direct UI manipulation, providing flexibility and enabling manual refinement of agent actions. Our user study, which included eight participants ranging from novices to proficient editors, demonstrated LAVE's effectiveness. The results also shed light on user perceptions of the proposed LLM-assisted editing paradigm and its impact on users' creativity and sense of co-creation. Based on these findings, we propose design implications to inform the future development of agent-assisted content editing.