OOTDiffusion：基于融合的潜在扩散，用于可控虚拟试穿

OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on

March 4, 2024

作者: Yuhao Xu, Tao Gu, Weifeng Chen, Chengcai Chen

cs.AI

摘要

基于图像的虚拟试穿（VTON）旨在生成一个穿着商店服装的目标人物图像，是一项具有挑战性的图像合成任务，不仅要求穿着的人物具有高保真度，还要完整保留服装细节。为了解决这个问题，我们提出了一种名为Outfitting over Try-on Diffusion（OOTDiffusion）的方法，利用预训练的潜在扩散模型的能力，并设计了一种新颖的网络架构，用于实现逼真且可控的虚拟试穿。在没有显式变形过程的情况下，我们提出了一种Outfitting UNet来学习服装细节特征，并通过我们提出的Outfitting Fusion将它们与目标人体融合在扩散模型的去噪过程中。为了进一步增强Outfitting UNet的可控性，我们在训练过程中引入了Outfitting Dropout，这使我们能够通过无分类器指导来调整服装特征的强度。我们在VITON-HD和Dress Code数据集上进行了全面实验，结果表明OOTDiffusion能够高效生成任意人物和服装图像的高质量穿着图像，其在保真度和可控性方面优于其他VTON方法，显示了虚拟试穿领域的令人印象深刻的突破。我们的源代码可在https://github.com/levihsu/OOTDiffusion 获取。

English

Image-based virtual try-on (VTON), which aims to generate an outfitted image of a target human wearing an in-shop garment, is a challenging image-synthesis task calling for not only high fidelity of the outfitted human but also full preservation of garment details. To tackle this issue, we propose Outfitting over Try-on Diffusion (OOTDiffusion), leveraging the power of pretrained latent diffusion models and designing a novel network architecture for realistic and controllable virtual try-on. Without an explicit warping process, we propose an outfitting UNet to learn the garment detail features, and merge them with the target human body via our proposed outfitting fusion in the denoising process of diffusion models. In order to further enhance the controllability of our outfitting UNet, we introduce outfitting dropout to the training process, which enables us to adjust the strength of garment features through classifier-free guidance. Our comprehensive experiments on the VITON-HD and Dress Code datasets demonstrate that OOTDiffusion efficiently generates high-quality outfitted images for arbitrary human and garment images, which outperforms other VTON methods in both fidelity and controllability, indicating an impressive breakthrough in virtual try-on. Our source code is available at https://github.com/levihsu/OOTDiffusion.