Garment3DGen：三維服裝風格化與紋理生成

Garment3DGen: 3D Garment Stylization and Texture Generation

March 27, 2024

作者: Nikolaos Sarafianos, Tuur Stuyck, Xiaoyu Xiang, Yilei Li, Jovan Popovic, Rakesh Ranjan

cs.AI

摘要

我們介紹了一種名為Garment3DGen的新方法，可以從基礎網格中合成3D服裝資產，並以單張輸入圖像作為指導。我們提出的方法允許用戶基於真實和合成圖像（如通過文本提示生成的圖像）生成3D紋理服裝。生成的資產可以直接應用於人體上進行布料模擬。首先，我們利用最近的圖像到3D擴散方法的進展來生成3D服裝幾何形狀。然而，由於這些幾何形狀不能直接用於下游任務，我們建議將它們作為虛擬地面實際值，並設置一個網格變形優化程序，將基礎模板網格變形以匹配生成的3D目標。其次，我們引入了精心設計的損失，使輸入的基礎網格可以自由變形到所需目標，同時保持網格質量和拓撲，以便進行模擬。最後，一個紋理估計模塊生成高保真度的紋理地圖，全局和局部一致，並忠實地捕捉輸入指導，使我們能夠渲染生成的3D資產。使用Garment3DGen，用戶可以生成所需的有紋理的3D服裝，無需藝術家干預。用戶可以提供描述所需生成的服裝的文本提示，以生成一個可進行模擬的3D資產。我們對各種真實和生成的資產進行了大量的定量和定性比較，並提供了如何生成可進行模擬的3D服裝的用例。

English

We introduce Garment3DGen a new method to synthesize 3D garment assets from a base mesh given a single input image as guidance. Our proposed approach allows users to generate 3D textured clothes based on both real and synthetic images, such as those generated by text prompts. The generated assets can be directly draped and simulated on human bodies. First, we leverage the recent progress of image to 3D diffusion methods to generate 3D garment geometries. However, since these geometries cannot be utilized directly for downstream tasks, we propose to use them as pseudo ground-truth and set up a mesh deformation optimization procedure that deforms a base template mesh to match the generated 3D target. Second, we introduce carefully designed losses that allow the input base mesh to freely deform towards the desired target, yet preserve mesh quality and topology such that they can be simulated. Finally, a texture estimation module generates high-fidelity texture maps that are globally and locally consistent and faithfully capture the input guidance, allowing us to render the generated 3D assets. With Garment3DGen users can generate the textured 3D garment of their choice without the need of artist intervention. One can provide a textual prompt describing the garment they desire to generate a simulation-ready 3D asset. We present a plethora of quantitative and qualitative comparisons on various assets both real and generated and provide use-cases of how one can generate simulation-ready 3D garments.