从单个角色图纸生成的3D动画：DrawingSpinUp

DrawingSpinUp: 3D Animation from Single Character Drawings

September 13, 2024

作者: Jie Zhou, Chufeng Xiao, Miu-Ling Lam, Hongbo Fu

cs.AI

摘要

将各种角色素描进行动画化是一项引人入胜的视觉内容创作任务。针对单个角色素描，现有的动画方法局限于平面2D运动，因此缺乏3D效果。另一种解决方案是从角色素描中重建3D模型作为代理，然后将3D运动数据重新定位到其中。然而，现有的图像到3D的方法在外观和几何方面无法很好地处理业余角色素描。我们观察到角色素描中常见的轮廓线会由于其视角相关性在纹理合成中引入显著的歧义。此外，由单线轮廓表示的细小区域（例如棍人的细肢）由于其精细结构而难以重建。为了解决这些问题，我们提出了一个新颖系统，DrawingSpinUp，以产生可信的3D动画，为角色素描注入生机，使其可以自由旋转、跳跃，甚至进行嘻哈舞蹈。为了改善外观，我们采用了一种先移除再恢复的策略，首先去除视角相关的轮廓线，然后在重新定位重建的角色后再渲染它们。为了改进几何结构，我们开发了基于骨架的细化变形算法，以完善由单线轮廓表示的细小结构。实验评估和感知用户研究表明，我们提出的方法优于现有的2D和3D动画方法，并能从单个角色素描生成高质量的3D动画。请参阅我们的项目页面（https://lordliang.github.io/DrawingSpinUp）获取代码和生成的动画。

English

Animating various character drawings is an engaging visual content creation task. Given a single character drawing, existing animation methods are limited to flat 2D motions and thus lack 3D effects. An alternative solution is to reconstruct a 3D model from a character drawing as a proxy and then retarget 3D motion data onto it. However, the existing image-to-3D methods could not work well for amateur character drawings in terms of appearance and geometry. We observe the contour lines, commonly existing in character drawings, would introduce significant ambiguity in texture synthesis due to their view-dependence. Additionally, thin regions represented by single-line contours are difficult to reconstruct (e.g., slim limbs of a stick figure) due to their delicate structures. To address these issues, we propose a novel system, DrawingSpinUp, to produce plausible 3D animations and breathe life into character drawings, allowing them to freely spin up, leap, and even perform a hip-hop dance. For appearance improvement, we adopt a removal-then-restoration strategy to first remove the view-dependent contour lines and then render them back after retargeting the reconstructed character. For geometry refinement, we develop a skeleton-based thinning deformation algorithm to refine the slim structures represented by the single-line contours. The experimental evaluations and a perceptual user study show that our proposed method outperforms the existing 2D and 3D animation methods and generates high-quality 3D animations from a single character drawing. Please refer to our project page (https://lordliang.github.io/DrawingSpinUp) for the code and generated animations.