Toon3D：從新的角度看卡通

Toon3D: Seeing Cartoons from a New Perspective

May 16, 2024

作者: Ethan Weber, Riley Peterlinz, Rohan Mathur, Frederik Warburg, Alexei A. Efros, Angjoo Kanazawa

cs.AI

摘要

在這項研究中，我們恢復了非幾何一致場景的潛在3D結構。我們專注於來自卡通和動畫的手繪圖像的分析。許多卡通是由藝術家創作而成，沒有使用3D渲染引擎，這意味著場景的任何新圖像都是手繪的。手繪圖像通常是對世界的忠實表現，但僅在質性上，因為人類很難一致地繪製物體或場景的多個透視圖。然而，人們可以輕鬆地從不一致的輸入中感知3D場景！在這項工作中，我們校正了2D繪製的不一致性，以恢復一個合理的3D結構，使新變形的圖像彼此一致。我們的流程包括一個用戶友好的標註工具、相機姿態估計和圖像變形，以恢復密集結構。我們的方法將圖像扭曲以遵循透視相機模型，使我們對齊的結果能夠被應用於新視角合成重建方法，體驗以前從未繪製過的卡通。我們的項目頁面是https://toon3d.studio/。

English

In this work, we recover the underlying 3D structure of non-geometrically consistent scenes. We focus our analysis on hand-drawn images from cartoons and anime. Many cartoons are created by artists without a 3D rendering engine, which means that any new image of a scene is hand-drawn. The hand-drawn images are usually faithful representations of the world, but only in a qualitative sense, since it is difficult for humans to draw multiple perspectives of an object or scene 3D consistently. Nevertheless, people can easily perceive 3D scenes from inconsistent inputs! In this work, we correct for 2D drawing inconsistencies to recover a plausible 3D structure such that the newly warped drawings are consistent with each other. Our pipeline consists of a user-friendly annotation tool, camera pose estimation, and image deformation to recover a dense structure. Our method warps images to obey a perspective camera model, enabling our aligned results to be plugged into novel-view synthesis reconstruction methods to experience cartoons from viewpoints never drawn before. Our project page is https://toon3d.studio/.