通过迭代反馈个性化扩散模型

摘要

在一个由机器学习推动视觉内容生成的时代，将人类反馈整合到生成模型中，为增强用户体验和输出质量提供了重要机遇。本研究探讨了将迭代人类反馈纳入基于扩散的文本到图像模型生成过程的策略。我们提出了FABRIC，这是一种无需训练的方法，适用于各种流行的扩散模型，利用了最常用架构中的自注意力层，以在一组反馈图像上调节扩散过程。为了严格评估我们的方法，我们引入了一种全面的评估方法，提供了一个强大的机制来量化整合人类反馈的生成视觉模型的性能。我们通过详尽分析展示，通过多轮迭代反馈，生成结果得到改善，从而隐式优化任意用户偏好。这些发现的潜在应用领域包括个性化内容创作和定制。

English

In an era where visual content generation is increasingly driven by machine learning, the integration of human feedback into generative models presents significant opportunities for enhancing user experience and output quality. This study explores strategies for incorporating iterative human feedback into the generative process of diffusion-based text-to-image models. We propose FABRIC, a training-free approach applicable to a wide range of popular diffusion models, which exploits the self-attention layer present in the most widely used architectures to condition the diffusion process on a set of feedback images. To ensure a rigorous assessment of our approach, we introduce a comprehensive evaluation methodology, offering a robust mechanism to quantify the performance of generative visual models that integrate human feedback. We show that generation results improve over multiple rounds of iterative feedback through exhaustive analysis, implicitly optimizing arbitrary user preferences. The potential applications of these findings extend to fields such as personalized content creation and customization.

通过迭代反馈个性化扩散模型

FABRIC: Personalizing Diffusion Models with Iterative Feedback

摘要

Support