MotionLCM：通過潛在一致性模型實現即時可控運動生成

MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model

April 30, 2024

作者: Wenxun Dai, Ling-Hao Chen, Jingbo Wang, Jinpeng Liu, Bo Dai, Yansong Tang

cs.AI

摘要

本研究介紹了MotionLCM，將可控運動生成擴展至實時水平。現有的文本條件運動生成中用於空間控制的方法存在顯著的運行時效率問題。為了解決這個問題，我們首先提出了運動潛在一致性模型（MotionLCM）用於運動生成，建立在潛在擴散模型（MLD）之上。通過採用一步（或少數步）推理，我們進一步提高了運動潛在擴散模型在運動生成中的運行時效率。為確保有效的可控性，我們在MotionLCM的潛在空間中加入了運動控制網（ControlNet），並在香草運動空間中啟用明確的控制信號（例如骨盆軌跡），以直接控制生成過程，類似於控制其他無潛在擴散模型用於運動生成。通過應用這些技術，我們的方法可以實時生成帶有文本和控制信號的人體運動。實驗結果展示了MotionLCM的卓越生成和控制能力，同時保持實時運行時效率。

English

This work introduces MotionLCM, extending controllable motion generation to a real-time level. Existing methods for spatial control in text-conditioned motion generation suffer from significant runtime inefficiency. To address this issue, we first propose the motion latent consistency model (MotionLCM) for motion generation, building upon the latent diffusion model (MLD). By employing one-step (or few-step) inference, we further improve the runtime efficiency of the motion latent diffusion model for motion generation. To ensure effective controllability, we incorporate a motion ControlNet within the latent space of MotionLCM and enable explicit control signals (e.g., pelvis trajectory) in the vanilla motion space to control the generation process directly, similar to controlling other latent-free diffusion models for motion generation. By employing these techniques, our approach can generate human motions with text and control signals in real-time. Experimental results demonstrate the remarkable generation and controlling capabilities of MotionLCM while maintaining real-time runtime efficiency.