從視頻擴散模型中提取關節運動學

摘要

我們提出了關節運動蒸餾（Articulated Kinematics Distillation, AKD）框架，該框架通過結合骨架動畫與現代生成模型的優勢，來生成高保真的角色動畫。AKD採用基於骨架的表示方法來處理綁定好的3D資產，通過專注於關節層級的控制，大幅減少了自由度（Degrees of Freedom, DoFs），從而實現了高效且一致的運動合成。借助預訓練的視頻擴散模型進行分數蒸餾採樣（Score Distillation Sampling, SDS），AKD在保持結構完整性的同時，蒸餾出複雜的關節運動，克服了4D神經變形場在保持形狀一致性方面所面臨的挑戰。此方法天然兼容基於物理的模擬，確保了物理上可信的交互。實驗表明，在文本到4D生成任務上，AKD相比現有工作展現出更優的3D一致性和運動質量。項目頁面：https://research.nvidia.com/labs/dir/akd/

English

We present Articulated Kinematics Distillation (AKD), a framework for generating high-fidelity character animations by merging the strengths of skeleton-based animation and modern generative models. AKD uses a skeleton-based representation for rigged 3D assets, drastically reducing the Degrees of Freedom (DoFs) by focusing on joint-level control, which allows for efficient, consistent motion synthesis. Through Score Distillation Sampling (SDS) with pre-trained video diffusion models, AKD distills complex, articulated motions while maintaining structural integrity, overcoming challenges faced by 4D neural deformation fields in preserving shape consistency. This approach is naturally compatible with physics-based simulation, ensuring physically plausible interactions. Experiments show that AKD achieves superior 3D consistency and motion quality compared with existing works on text-to-4D generation. Project page: https://research.nvidia.com/labs/dir/akd/

從視頻擴散模型中提取關節運動學

Articulated Kinematics Distillation from Video Diffusion Models

摘要

Support