深度几何化卡通线插帧

Deep Geometrized Cartoon Line Inbetweening

September 28, 2023

作者: Li Siyao, Tianpei Gu, Weiye Xiao, Henghui Ding, Ziwei Liu, Chen Change Loy

cs.AI

摘要

我们的目标是解决动漫行业中一个重要但研究不足的问题，即卡通线描的中间绘制。中间绘制涉及在两幅黑白线描之间生成中间帧，这是一项耗时且昂贵的过程，可以从自动化中受益。然而，现有依赖于匹配和整体栅格图像变形的帧插值方法不适用于线描的中间绘制，并且常常会产生破坏复杂线结构的模糊伪影。为了保留线描的精确性和细节，我们提出了一种新方法，AnimeInbet，将栅格线描几何化为端点图，并将中间绘制任务重新构建为一个具有顶点重新定位的图融合问题。我们的方法能够有效捕捉线描的稀疏性和独特结构，在中间绘制过程中保留细节。这得益于我们的新颖模块，即顶点几何嵌入、顶点对应Transformer、顶点重新定位的有效机制以及可见性预测器。为了训练我们的方法，我们引入了MixamoLine240，一个具有地面真实矢量化和匹配标签的新线描数据集。我们的实验证明，AnimeInbet合成了高质量、干净且完整的中间线描，从数量和质量上优于现有方法，尤其是在有大幅度运动的情况下。数据和代码可在https://github.com/lisiyao21/AnimeInbet获得。

English

We aim to address a significant but understudied problem in the anime industry, namely the inbetweening of cartoon line drawings. Inbetweening involves generating intermediate frames between two black-and-white line drawings and is a time-consuming and expensive process that can benefit from automation. However, existing frame interpolation methods that rely on matching and warping whole raster images are unsuitable for line inbetweening and often produce blurring artifacts that damage the intricate line structures. To preserve the precision and detail of the line drawings, we propose a new approach, AnimeInbet, which geometrizes raster line drawings into graphs of endpoints and reframes the inbetweening task as a graph fusion problem with vertex repositioning. Our method can effectively capture the sparsity and unique structure of line drawings while preserving the details during inbetweening. This is made possible via our novel modules, i.e., vertex geometric embedding, a vertex correspondence Transformer, an effective mechanism for vertex repositioning and a visibility predictor. To train our method, we introduce MixamoLine240, a new dataset of line drawings with ground truth vectorization and matching labels. Our experiments demonstrate that AnimeInbet synthesizes high-quality, clean, and complete intermediate line drawings, outperforming existing methods quantitatively and qualitatively, especially in cases with large motions. Data and code are available at https://github.com/lisiyao21/AnimeInbet.