CrowdMoGen：零样本文本驱动的集体运动生成

CrowdMoGen: Zero-Shot Text-Driven Collective Motion Generation

July 8, 2024

作者: Xinying Guo, Mingyuan Zhang, Haozhe Xie, Chenyang Gu, Ziwei Liu

cs.AI

摘要

人群运动生成在娱乐行业（如动画和游戏）以及战略领域（如城市模拟和规划）中至关重要。这项新任务需要精细地整合控制和生成，以在特定空间和语义约束下实现逼真地合成人群动态，其挑战尚未完全探索。一方面，现有的人类运动生成模型通常侧重于个体行为，忽视了集体行为的复杂性。另一方面，最近的多人运动生成方法严重依赖预定义场景，并且仅限于固定且有限的人际互动数量，从而限制了它们的实用性。为了克服这些挑战，我们引入了CrowdMoGen，这是一个零样本文本驱动框架，利用大型语言模型（LLM）的力量将集体智慧整合到运动生成框架中作为指导，从而实现人群运动的通用规划和生成，而无需配对训练数据。我们的框架包括两个关键组件：1）人群场景规划器，根据特定场景背景或引入的扰动学习协调运动和动态，以及2）集体运动生成器，根据整体计划高效合成所需的集体运动。广泛的定量和定性实验证实了我们框架的有效性，它不仅通过提供可扩展和通用的解决方案填补了人群运动生成任务的关键空白，而且实现了高水平的逼真性和灵活性。

English

Crowd Motion Generation is essential in entertainment industries such as animation and games as well as in strategic fields like urban simulation and planning. This new task requires an intricate integration of control and generation to realistically synthesize crowd dynamics under specific spatial and semantic constraints, whose challenges are yet to be fully explored. On the one hand, existing human motion generation models typically focus on individual behaviors, neglecting the complexities of collective behaviors. On the other hand, recent methods for multi-person motion generation depend heavily on pre-defined scenarios and are limited to a fixed, small number of inter-person interactions, thus hampering their practicality. To overcome these challenges, we introduce CrowdMoGen, a zero-shot text-driven framework that harnesses the power of Large Language Model (LLM) to incorporate the collective intelligence into the motion generation framework as guidance, thereby enabling generalizable planning and generation of crowd motions without paired training data. Our framework consists of two key components: 1) Crowd Scene Planner that learns to coordinate motions and dynamics according to specific scene contexts or introduced perturbations, and 2) Collective Motion Generator that efficiently synthesizes the required collective motions based on the holistic plans. Extensive quantitative and qualitative experiments have validated the effectiveness of our framework, which not only fills a critical gap by providing scalable and generalizable solutions for Crowd Motion Generation task but also achieves high levels of realism and flexibility.