SketchMetaFace：一個基於學習的素描界面，用於高保真度的3D角色臉部建模。

SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling

July 3, 2023

作者: Zhongjin Luo, Dong Du, Heming Zhu, Yizhou Yu, Hongbo Fu, Xiaoguang Han

cs.AI

摘要

建模3D頭像對於AR/VR、遊戲和拍攝等各種應用場景都有好處。角色臉部作為頭像的重要組成部分，為頭像帶來了顯著的多樣性和生動性。然而，構建3D角色臉部模型通常需要使用商業工具，即使對於有經驗的藝術家來說也是一項繁重的工作。各種現有的基於草圖的工具無法支持業餘者建模多樣的面部形狀和豐富的幾何細節。在本文中，我們提出了SketchMetaFace - 一個針對業餘用戶設計的草圖系統，可在幾分鐘內建模高保真度的3D臉部。我們精心設計了用戶界面和底層算法。首先，我們採用了曲率感知筆劃，以更好地支持雕刻面部細節的可控性。其次，考慮到將2D草圖映射到3D模型的關鍵問題，我們開發了一種名為“隱式和深度引導網格建模”（IDGMM）的新型基於學習的方法。它融合了網格、隱式和深度表示的優勢，實現了高質量和高效率的結果。此外，為了進一步支持可用性，我們提出了一種從粗到細的2D草圖界面設計和一個數據驅動的筆劃建議工具。用戶研究表明，我們的系統在易用性和視覺質量方面優於現有的建模工具。實驗分析還表明，IDGMM在精度和效率之間取得了更好的折衷。SketchMetaFace可在https://zhongjinluo.github.io/SketchMetaFace/ 上獲得。

English

Modeling 3D avatars benefits various application scenarios such as AR/VR, gaming, and filming. Character faces contribute significant diversity and vividity as a vital component of avatars. However, building 3D character face models usually requires a heavy workload with commercial tools, even for experienced artists. Various existing sketch-based tools fail to support amateurs in modeling diverse facial shapes and rich geometric details. In this paper, we present SketchMetaFace - a sketching system targeting amateur users to model high-fidelity 3D faces in minutes. We carefully design both the user interface and the underlying algorithm. First, curvature-aware strokes are adopted to better support the controllability of carving facial details. Second, considering the key problem of mapping a 2D sketch map to a 3D model, we develop a novel learning-based method termed "Implicit and Depth Guided Mesh Modeling" (IDGMM). It fuses the advantages of mesh, implicit, and depth representations to achieve high-quality results with high efficiency. In addition, to further support usability, we present a coarse-to-fine 2D sketching interface design and a data-driven stroke suggestion tool. User studies demonstrate the superiority of our system over existing modeling tools in terms of the ease to use and visual quality of results. Experimental analyses also show that IDGMM reaches a better trade-off between accuracy and efficiency. SketchMetaFace are available at https://zhongjinluo.github.io/SketchMetaFace/.