Garment3DGen：三维服装风格化与纹理生成

Garment3DGen: 3D Garment Stylization and Texture Generation

March 27, 2024

作者: Nikolaos Sarafianos, Tuur Stuyck, Xiaoyu Xiang, Yilei Li, Jovan Popovic, Rakesh Ranjan

cs.AI

摘要

我们介绍了Garment3DGen，这是一种新方法，可以根据单个输入图像生成3D服装模型。我们提出的方法允许用户基于真实和合成图像（例如通过文本提示生成的图像）生成3D纹理服装。生成的模型可以直接覆盖在人体上并进行模拟。首先，我们利用最近的图像到3D扩散方法的进展来生成3D服装几何结构。然而，由于这些几何结构不能直接用于下游任务，我们建议将它们作为伪地面真实值，并设置一个网格变形优化过程，将基础模板网格变形以匹配生成的3D目标。其次，我们引入了精心设计的损失函数，允许输入的基础网格自由变形到期望的目标，同时保持网格质量和拓扑结构，以便进行模拟。最后，一个纹理估计模块生成高保真度的纹理贴图，这些贴图在全局和局部上保持一致，并忠实地捕捉输入的指导，从而使我们能够渲染生成的3D模型。使用Garment3DGen，用户可以生成所需的带纹理的3D服装，无需艺术家干预。用户可以提供描述他们想要生成的服装的文本提示，以生成一个可用于模拟的3D模型。我们展示了对各种真实和生成的模型进行的大量定量和定性比较，并提供了如何生成可用于模拟的3D服装的用例。

English

We introduce Garment3DGen a new method to synthesize 3D garment assets from a base mesh given a single input image as guidance. Our proposed approach allows users to generate 3D textured clothes based on both real and synthetic images, such as those generated by text prompts. The generated assets can be directly draped and simulated on human bodies. First, we leverage the recent progress of image to 3D diffusion methods to generate 3D garment geometries. However, since these geometries cannot be utilized directly for downstream tasks, we propose to use them as pseudo ground-truth and set up a mesh deformation optimization procedure that deforms a base template mesh to match the generated 3D target. Second, we introduce carefully designed losses that allow the input base mesh to freely deform towards the desired target, yet preserve mesh quality and topology such that they can be simulated. Finally, a texture estimation module generates high-fidelity texture maps that are globally and locally consistent and faithfully capture the input guidance, allowing us to render the generated 3D assets. With Garment3DGen users can generate the textured 3D garment of their choice without the need of artist intervention. One can provide a textual prompt describing the garment they desire to generate a simulation-ready 3D asset. We present a plethora of quantitative and qualitative comparisons on various assets both real and generated and provide use-cases of how one can generate simulation-ready 3D garments.