CharacterShot：可控且一致的4D角色动画

摘要

本文提出CharacterShot，一个可控且一致的4D角色动画框架，使任何设计师都能从单一参考角色图像和2D姿态序列中创建动态3D角色（即4D角色动画）。我们首先基于前沿的DiT图像到视频模型预训练一个强大的2D角色动画模型，该模型允许任何2D姿态序列作为可控信号。随后，通过引入双注意力模块并结合相机先验，我们将动画模型从2D提升至3D，生成具有时空一致性和空间视角一致性的多视角视频。最后，我们对这些多视角视频采用新颖的邻域约束4D高斯溅射优化，得到连续稳定的4D角色表示。此外，为提升角色中心性能，我们构建了一个大规模数据集Character4D，包含13,115个具有多样外观和动作的独特角色，从多个视角渲染而成。在我们新构建的基准测试CharacterBench上的大量实验表明，我们的方法优于当前最先进的技术。代码、模型和数据集将在https://github.com/Jeoyal/CharacterShot 公开提供。

English

In this paper, we propose CharacterShot, a controllable and consistent 4D character animation framework that enables any individual designer to create dynamic 3D characters (i.e., 4D character animation) from a single reference character image and a 2D pose sequence. We begin by pretraining a powerful 2D character animation model based on a cutting-edge DiT-based image-to-video model, which allows for any 2D pose sequnce as controllable signal. We then lift the animation model from 2D to 3D through introducing dual-attention module together with camera prior to generate multi-view videos with spatial-temporal and spatial-view consistency. Finally, we employ a novel neighbor-constrained 4D gaussian splatting optimization on these multi-view videos, resulting in continuous and stable 4D character representations. Moreover, to improve character-centric performance, we construct a large-scale dataset Character4D, containing 13,115 unique characters with diverse appearances and motions, rendered from multiple viewpoints. Extensive experiments on our newly constructed benchmark, CharacterBench, demonstrate that our approach outperforms current state-of-the-art methods. Code, models, and datasets will be publicly available at https://github.com/Jeoyal/CharacterShot.

CharacterShot：可控且一致的4D角色动画

CharacterShot: Controllable and Consistent 4D Character Animation

摘要

Support