零样本主题驱动生成中的负向引导主题保真度优化

Negative-Guided Subject Fidelity Optimization for Zero-Shot Subject-Driven Generation

June 4, 2025

作者: Chaehun Shin, Jooyoung Choi, Johan Barthelemy, Jungbeom Lee, Sungroh Yoon

cs.AI

摘要

我们提出了主题保真度优化（Subject Fidelity Optimization, SFO），这是一种新颖的对比学习框架，专为零样本主题驱动生成而设计，旨在提升主题保真度。与仅依赖正样本目标并在预训练阶段使用扩散损失的监督微调方法不同，SFO引入了合成负样本目标，并通过成对比较明确引导模型偏好正样本而非负样本。针对负样本，我们提出了条件退化负采样（Condition-Degradation Negative Sampling, CDNS），该方法通过有意降低视觉和文本线索的完整性，自动生成具有区分性和信息量的负样本，而无需昂贵的人工标注。此外，我们重新加权扩散时间步，将微调重点放在主题细节显现的中间步骤上。大量实验表明，在主题驱动生成基准测试中，结合CDNS的SFO在主题保真度和文本对齐方面均显著优于基线方法。项目页面：https://subjectfidelityoptimization.github.io/

English

We present Subject Fidelity Optimization (SFO), a novel comparative learning framework for zero-shot subject-driven generation that enhances subject fidelity. Beyond supervised fine-tuning methods that rely only on positive targets and use the diffusion loss as in the pre-training stage, SFO introduces synthetic negative targets and explicitly guides the model to favor positives over negatives through pairwise comparison. For negative targets, we propose Condition-Degradation Negative Sampling (CDNS), which automatically generates distinctive and informative negatives by intentionally degrading visual and textual cues without expensive human annotations. Moreover, we reweight the diffusion timesteps to focus finetuning on intermediate steps where subject details emerge. Extensive experiments demonstrate that SFO with CDNS significantly outperforms baselines in terms of both subject fidelity and text alignment on a subject-driven generation benchmark. Project page: https://subjectfidelityoptimization.github.io/