MotionLCM：基于潜在一致性模型的实时可控运动生成

MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model

April 30, 2024

作者: Wenxun Dai, Ling-Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, Yansong Tang

cs.AI

摘要

本文介绍了MotionLCM，将可控动作生成扩展到实时水平。现有的基于文本条件的空间控制动作生成方法存在显著的运行时低效性。为了解决这个问题，我们首先提出了运动潜在一致性模型（MotionLCM）用于动作生成，建立在潜在扩散模型（MLD）的基础上。通过采用一步（或少步）推断，我们进一步提高了用于动作生成的运动潜在扩散模型的运行时效率。为了确保有效的可控性，我们在MotionLCM的潜在空间中加入了一个运动控制网络（ControlNet），并在香草动作空间中启用显式控制信号（例如骨盆轨迹）来直接控制生成过程，类似于控制其他无潜在扩散模型用于动作生成。通过采用这些技术，我们的方法可以实时生成带有文本和控制信号的人类动作。实验结果展示了MotionLCM的显著生成和控制能力，同时保持实时运行时效率。

English

This work introduces MotionLCM, extending controllable motion generation to a real-time level. Existing methods for spatial control in text-conditioned motion generation suffer from significant runtime inefficiency. To address this issue, we first propose the motion latent consistency model (MotionLCM) for motion generation, building upon the latent diffusion model (MLD). By employing one-step (or few-step) inference, we further improve the runtime efficiency of the motion latent diffusion model for motion generation. To ensure effective controllability, we incorporate a motion ControlNet within the latent space of MotionLCM and enable explicit control signals (e.g., pelvis trajectory) in the vanilla motion space to control the generation process directly, similar to controlling other latent-free diffusion models for motion generation. By employing these techniques, our approach can generate human motions with text and control signals in real-time. Experimental results demonstrate the remarkable generation and controlling capabilities of MotionLCM while maintaining real-time runtime efficiency.