ReMoMask：檢索增強型遮罩動作生成

摘要

文本到動作（T2M）生成旨在從自然語言描述中合成出真實且語義對齊的人體動作序列。然而，現有方法面臨雙重挑戰：生成模型（如擴散模型）存在多樣性有限、錯誤累積和物理不可行性等問題，而檢索增強生成（RAG）方法則表現出擴散慣性、部分模式崩潰和異步偽影等缺陷。為解決這些限制，我們提出了ReMoMask，這是一個整合了三項關鍵創新的統一框架：1）雙向動量文本-動作模型通過動量隊列將負樣本規模與批次大小解耦，顯著提高了跨模態檢索精度；2）語義時空注意力機制在部分級融合過程中施加生物力學約束，以消除異步偽影；3）RAG-無分類器指導結合少量無條件生成以增強泛化能力。基於MoMask的RVQ-VAE，ReMoMask能夠以最少的步驟高效生成時間連貫的動作。在標準基準上的大量實驗表明，ReMoMask達到了最先進的性能，與之前的SOTA方法RAG-T2M相比，在HumanML3D和KIT-ML上的FID分數分別提高了3.88%和10.97%。代碼：https://github.com/AIGeeksGroup/ReMoMask。網站：https://aigeeksgroup.github.io/ReMoMask。

English

Text-to-Motion (T2M) generation aims to synthesize realistic and semantically aligned human motion sequences from natural language descriptions. However, current approaches face dual challenges: Generative models (e.g., diffusion models) suffer from limited diversity, error accumulation, and physical implausibility, while Retrieval-Augmented Generation (RAG) methods exhibit diffusion inertia, partial-mode collapse, and asynchronous artifacts. To address these limitations, we propose ReMoMask, a unified framework integrating three key innovations: 1) A Bidirectional Momentum Text-Motion Model decouples negative sample scale from batch size via momentum queues, substantially improving cross-modal retrieval precision; 2) A Semantic Spatio-temporal Attention mechanism enforces biomechanical constraints during part-level fusion to eliminate asynchronous artifacts; 3) RAG-Classier-Free Guidance incorporates minor unconditional generation to enhance generalization. Built upon MoMask's RVQ-VAE, ReMoMask efficiently generates temporally coherent motions in minimal steps. Extensive experiments on standard benchmarks demonstrate the state-of-the-art performance of ReMoMask, achieving a 3.88% and 10.97% improvement in FID scores on HumanML3D and KIT-ML, respectively, compared to the previous SOTA method RAG-T2M. Code: https://github.com/AIGeeksGroup/ReMoMask. Website: https://aigeeksgroup.github.io/ReMoMask.

ReMoMask：檢索增強型遮罩動作生成

ReMoMask: Retrieval-Augmented Masked Motion Generation

摘要

Support