RectifID:通过锚定分类器指导个性化修正流。
RectifID: Personalizing Rectified Flow with Anchored Classifier Guidance
May 23, 2024
作者: Zhicheng Sun, Zhenhao Yang, Yang Jin, Haozhe Chi, Kun Xu, Kun Xu, Liwei Chen, Hao Jiang, Di Zhang, Yang Song, Kun Gai, Yadong Mu
cs.AI
摘要
定制扩散模型以从用户提供的参考图像生成保持身份的图像是一个引人入胜的新问题。目前的方法通常需要在大量领域特定图像上进行训练,以实现身份保留,但在不同用例之间缺乏灵活性。为了解决这个问题,我们利用分类器引导,这是一种无需训练的技术,通过使用现有的分类器来引导扩散模型,实现个性化图像生成。我们的研究表明,基于最近的修正流框架,普通分类器引导在需要特殊分类器方面的主要限制可以通过简单的固定点解决方案来解决,从而允许使用现成的图像判别器进行灵活个性化。此外,基于参考流轨迹的求解过程被证明是稳定的,并具有收敛保证。所得方法在具有不同现成图像判别器的修正流上实现,为人脸、实时主体和某些对象提供了有利的个性化结果。代码可在https://github.com/feifeiobama/RectifID找到。
English
Customizing diffusion models to generate identity-preserving images from
user-provided reference images is an intriguing new problem. The prevalent
approaches typically require training on extensive domain-specific images to
achieve identity preservation, which lacks flexibility across different use
cases. To address this issue, we exploit classifier guidance, a training-free
technique that steers diffusion models using an existing classifier, for
personalized image generation. Our study shows that based on a recent rectified
flow framework, the major limitation of vanilla classifier guidance in
requiring a special classifier can be resolved with a simple fixed-point
solution, allowing flexible personalization with off-the-shelf image
discriminators. Moreover, its solving procedure proves to be stable when
anchored to a reference flow trajectory, with a convergence guarantee. The
derived method is implemented on rectified flow with different off-the-shelf
image discriminators, delivering advantageous personalization results for human
faces, live subjects, and certain objects. Code is available at
https://github.com/feifeiobama/RectifID.