UniFL：通过统一反馈学习改善稳定扩散

摘要

扩散模型已经彻底改变了图像生成领域，导致高质量模型和多样化下游应用的大量涌现。然而，尽管取得了重大进展，当前竞争性解决方案仍然存在几个限制，包括视觉质量较差、缺乏审美吸引力以及推理效率低下，而且目前尚无全面解决方案。为了解决这些挑战，我们提出了UniFL，这是一个利用反馈学习全面增强扩散模型的统一框架。UniFL以其通用、有效和可泛化的特点脱颖而出，适用于各种扩散模型，如SD1.5和SDXL。值得注意的是，UniFL包含三个关键组成部分：感知反馈学习，用于增强视觉质量；解耦反馈学习，用于提高审美吸引力；对抗反馈学习，用于优化推理速度。深入实验和广泛用户研究验证了我们提出的方法在提升生成模型质量和加速方面的卓越性能。例如，UniFL在生成质量方面比ImageReward高出17%的用户偏好，并且在4步推理中的性能超过了LCM和SDXL Turbo分别为57%和20%。此外，我们已经验证了我们的方法在包括Lora、ControlNet和AnimateDiff在内的下游任务中的有效性。

English

Diffusion models have revolutionized the field of image generation, leading to the proliferation of high-quality models and diverse downstream applications. However, despite these significant advancements, the current competitive solutions still suffer from several limitations, including inferior visual quality, a lack of aesthetic appeal, and inefficient inference, without a comprehensive solution in sight. To address these challenges, we present UniFL, a unified framework that leverages feedback learning to enhance diffusion models comprehensively. UniFL stands out as a universal, effective, and generalizable solution applicable to various diffusion models, such as SD1.5 and SDXL. Notably, UniFL incorporates three key components: perceptual feedback learning, which enhances visual quality; decoupled feedback learning, which improves aesthetic appeal; and adversarial feedback learning, which optimizes inference speed. In-depth experiments and extensive user studies validate the superior performance of our proposed method in enhancing both the quality of generated models and their acceleration. For instance, UniFL surpasses ImageReward by 17% user preference in terms of generation quality and outperforms LCM and SDXL Turbo by 57% and 20% in 4-step inference. Moreover, we have verified the efficacy of our approach in downstream tasks, including Lora, ControlNet, and AnimateDiff.

UniFL：通过统一反馈学习改善稳定扩散

UniFL: Improve Stable Diffusion via Unified Feedback Learning

摘要

Summary

Support

Support