弗蘭肯運動：部件級人體動作生成與組合

摘要

近年來，基於文本提示的人體動作生成技術取得了顯著進展。然而，由於缺乏細粒度、部位層級的動作標註，現有方法主要依賴序列層級或動作層級的描述，這限制了個別身體部位的可控性。本研究利用大型語言模型的推理能力，構建了一個具有原子化且具時間感知的部位層級文本標註之高品質動作資料集。有別於先前僅提供固定時間段同步部位標註或僅依賴全局序列標籤的資料集，我們的資料集以精細時間解析度捕捉非同步且語義獨立的部位運動。基於此資料集，我們提出一個基於擴散模型的部位感知動作生成框架FrankenMotion，其中每個身體部位由其專屬的時間結構化文本提示引導。據我們所知，這是首個提供原子化時間感知部位層級動作標註，並實現兼具空間（身體部位）與時間（原子化動作）控制之動作生成模型的研究。實驗表明，FrankenMotion在針對我們設定進行改編與重新訓練的所有基準模型中表現最優，且能組合出訓練時未見過的動作。我們的程式碼與資料集將於論文發表時公開釋出。

English

Human motion generation from text prompts has made remarkable progress in recent years. However, existing methods primarily rely on either sequence-level or action-level descriptions due to the absence of fine-grained, part-level motion annotations. This limits their controllability over individual body parts. In this work, we construct a high-quality motion dataset with atomic, temporally-aware part-level text annotations, leveraging the reasoning capabilities of large language models (LLMs). Unlike prior datasets that either provide synchronized part captions with fixed time segments or rely solely on global sequence labels, our dataset captures asynchronous and semantically distinct part movements at fine temporal resolution. Based on this dataset, we introduce a diffusion-based part-aware motion generation framework, namely FrankenMotion, where each body part is guided by its own temporally-structured textual prompt. This is, to our knowledge, the first work to provide atomic, temporally-aware part-level motion annotations and have a model that allows motion generation with both spatial (body part) and temporal (atomic action) control. Experiments demonstrate that FrankenMotion outperforms all previous baseline models adapted and retrained for our setting, and our model can compose motions unseen during training. Our code and dataset will be publicly available upon publication.

弗蘭肯運動：部件級人體動作生成與組合

FrankenMotion: Part-level Human Motion Generation and Composition

摘要

Support