生成式抠图

Matting by Generation

July 30, 2024

作者: Zhixiang Wang, Baiang Li, Jian Wang, Yu-Lun Liu, Jinwei Gu, Yung-Yu Chuang, Shin'ichi Satoh

cs.AI

摘要

本文介绍了一种创新的图像抠图方法，将传统的基于回归的任务重新定义为生成建模挑战。我们的方法利用潜在扩散模型的能力，结合丰富的预训练知识，对抠图过程进行规范化。我们提出了新颖的架构创新，使我们的模型能够生成分辨率和细节更出色的抠图。所提出的方法多才多艺，可以执行无引导和基于引导的图像抠图，适应各种额外线索。我们在三个基准数据集上进行了全面评估，展示了我们方法在定量和定性上的卓越性能。结果不仅反映了我们方法的强大有效性，还突出了其生成视觉上引人注目、接近照片般逼真质量的抠图的能力。本文的项目页面位于https://lightchaserx.github.io/matting-by-generation/。

English

This paper introduces an innovative approach for image matting that redefines the traditional regression-based task as a generative modeling challenge. Our method harnesses the capabilities of latent diffusion models, enriched with extensive pre-trained knowledge, to regularize the matting process. We present novel architectural innovations that empower our model to produce mattes with superior resolution and detail. The proposed method is versatile and can perform both guidance-free and guidance-based image matting, accommodating a variety of additional cues. Our comprehensive evaluation across three benchmark datasets demonstrates the superior performance of our approach, both quantitatively and qualitatively. The results not only reflect our method's robust effectiveness but also highlight its ability to generate visually compelling mattes that approach photorealistic quality. The project page for this paper is available at https://lightchaserx.github.io/matting-by-generation/