ResAdapter：用于扩散模型的领域一致的分辨率适配器
ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models
March 4, 2024
作者: Jiaxiang Cheng, Pan Xie, Xin Xia, Jiashi Li, Jie Wu, Yuxi Ren, Huixia Li, Xuefeng Xiao, Min Zheng, Lean Fu
cs.AI
摘要
最近在文本到图像模型（例如，稳定扩散）和相应的个性化技术（例如，DreamBooth和LoRA）方面取得了进展，使个人能够生成高质量且富有想象力的图像。然而，在生成分辨率超出其训练领域的图像时，它们经常受到限制。为了克服这一限制，我们提出了分辨率适配器（ResAdapter），这是一种专为扩散模型设计的领域一致适配器，可生成具有无限制分辨率和宽高比的图像。与处理静态分辨率图像并进行复杂后处理操作的其他多分辨率生成方法不同，ResAdapter直接生成具有动态分辨率的图像。特别是，在深入了解纯分辨率先验之后，ResAdapter在通用数据集上训练，使用个性化扩散模型生成无分辨率限制的图像，同时保留其原始风格领域。全面的实验表明，ResAdapter仅需0.5M即可处理任意扩散模型的灵活分辨率图像。更广泛的实验表明，ResAdapter与其他模块（例如ControlNet、IP-Adapter和LCM-LoRA）兼容，可用于跨广泛分辨率范围生成图像，并可集成到其他多分辨率模型（例如ElasticDiffusion）中，以高效生成更高分辨率的图像。项目链接为https://res-adapter.github.io。
English
Recent advancement in text-to-image models (e.g., Stable Diffusion) and
corresponding personalized technologies (e.g., DreamBooth and LoRA) enables
individuals to generate high-quality and imaginative images. However, they
often suffer from limitations when generating images with resolutions outside
of their trained domain. To overcome this limitation, we present the Resolution
Adapter (ResAdapter), a domain-consistent adapter designed for diffusion models
to generate images with unrestricted resolutions and aspect ratios. Unlike
other multi-resolution generation methods that process images of static
resolution with complex post-process operations, ResAdapter directly generates
images with the dynamical resolution. Especially, after learning a deep
understanding of pure resolution priors, ResAdapter trained on the general
dataset, generates resolution-free images with personalized diffusion models
while preserving their original style domain. Comprehensive experiments
demonstrate that ResAdapter with only 0.5M can process images with flexible
resolutions for arbitrary diffusion models. More extended experiments
demonstrate that ResAdapter is compatible with other modules (e.g., ControlNet,
IP-Adapter and LCM-LoRA) for image generation across a broad range of
resolutions, and can be integrated into other multi-resolution model (e.g.,
ElasticDiffusion) for efficiently generating higher-resolution images. Project
link is https://res-adapter.github.io