FaceChain-SuDe：构建派生类以继承类别属性，用于一次性主题驱动生成。

FaceChain-SuDe: Building Derived Class to Inherit Category Attributes for One-shot Subject-Driven Generation

March 11, 2024

作者: Pengchong Qiao, Lei Shang, Chang Liu, Baigui Sun, Xiangyang Ji, Jie Chen

cs.AI

摘要

最近，基于主题驱动的生成引起了广泛关注，因为它能够个性化文本到图像的生成。典型的研究侧重于学习新主题的私有属性。然而，一个重要的事实未被认真对待，即主题不是一个孤立的新概念，而应是预训练模型中某一类别的专门化。这导致主题未能全面继承其类别中的属性，导致属性相关生成质量不佳。在本文中，受面向对象编程的启发，我们将主题建模为一个派生类，其基类是其语义类别。这种建模使主题能够从其类别中继承公共属性，同时从用户提供的示例中学习其私有属性。具体而言，我们提出了一种即插即用的方法，名为主题派生正则化（SuDe）。它通过约束主题驱动生成的图像在语义上属于主题的类别，构建了基础派生类建模。在各种主题上进行的大量实验，基于三种基线和两种主干网络，表明我们的SuDe能够实现富有想象力的属性相关生成，同时保持主题的忠实性。代码将很快在FaceChain（https://github.com/modelscope/facechain）上开源。

English

Subject-driven generation has garnered significant interest recently due to its ability to personalize text-to-image generation. Typical works focus on learning the new subject's private attributes. However, an important fact has not been taken seriously that a subject is not an isolated new concept but should be a specialization of a certain category in the pre-trained model. This results in the subject failing to comprehensively inherit the attributes in its category, causing poor attribute-related generations. In this paper, motivated by object-oriented programming, we model the subject as a derived class whose base class is its semantic category. This modeling enables the subject to inherit public attributes from its category while learning its private attributes from the user-provided example. Specifically, we propose a plug-and-play method, Subject-Derived regularization (SuDe). It constructs the base-derived class modeling by constraining the subject-driven generated images to semantically belong to the subject's category. Extensive experiments under three baselines and two backbones on various subjects show that our SuDe enables imaginative attribute-related generations while maintaining subject fidelity. Codes will be open sourced soon at FaceChain (https://github.com/modelscope/facechain).