FaceChain-SuDe：建立衍生類別以繼承類別屬性，用於一次性主題驅動生成。

FaceChain-SuDe: Building Derived Class to Inherit Category Attributes for One-shot Subject-Driven Generation

March 11, 2024

作者: Pengchong Qiao, Lei Shang, Chang Liu, Baigui Sun, Xiangyang Ji, Jie Chen

cs.AI

摘要

最近，基於主題驅動的生成技術引起了廣泛關注，因為它能夠個性化文本到圖像的生成。典型的研究著重於學習新主題的私有屬性。然而，一個重要的事實沒有得到足夠重視，即主題並不是一個孤立的新概念，而應該是預先訓練模型中某個類別的特化。這導致主題無法全面繼承其類別中的屬性，導致屬性相關生成品質不佳。本文受物件導向編程啟發，將主題建模為一個衍生類別，其基礎類別是其語義類別。這種建模使主題能夠從其類別中繼承公共屬性，同時從用戶提供的示例中學習其私有屬性。具體來說，我們提出了一種即插即用的方法，名為主題衍生正則化（SuDe）。通過將主題驅動生成的圖像限制為語義上屬於主題類別，來構建基礎衍生類別建模。在各種主題上進行的大量實驗，基於三個基準和兩個骨幹，顯示我們的 SuDe 能夠實現具想像力的屬性相關生成，同時保持主題的忠實性。代碼將很快在 FaceChain（https://github.com/modelscope/facechain）上開源。

English

Subject-driven generation has garnered significant interest recently due to its ability to personalize text-to-image generation. Typical works focus on learning the new subject's private attributes. However, an important fact has not been taken seriously that a subject is not an isolated new concept but should be a specialization of a certain category in the pre-trained model. This results in the subject failing to comprehensively inherit the attributes in its category, causing poor attribute-related generations. In this paper, motivated by object-oriented programming, we model the subject as a derived class whose base class is its semantic category. This modeling enables the subject to inherit public attributes from its category while learning its private attributes from the user-provided example. Specifically, we propose a plug-and-play method, Subject-Derived regularization (SuDe). It constructs the base-derived class modeling by constraining the subject-driven generated images to semantically belong to the subject's category. Extensive experiments under three baselines and two backbones on various subjects show that our SuDe enables imaginative attribute-related generations while maintaining subject fidelity. Codes will be open sourced soon at FaceChain (https://github.com/modelscope/facechain).