ResAdapter：用於擴散模型的領域一致解析適配器
ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models
March 4, 2024
作者: Jiaxiang Cheng, Pan Xie, Xin Xia, Jiashi Li, Jie Wu, Yuxi Ren, Huixia Li, Xuefeng Xiao, Min Zheng, Lean Fu
cs.AI
摘要
最近在文本到圖像模型方面的進展（例如穩定擴散）以及相應的個性化技術（例如DreamBooth和LoRA）使個人能夠生成高質量且富有想像力的圖像。然而，在生成分辨率超出其訓練領域的圖像時，這些模型通常會受到限制。為了克服這一限制，我們提出了解析度適配器（ResAdapter），這是一個為擴散模型設計的領域一致適配器，可生成具有無限制分辨率和長寬比的圖像。與其他多分辨率生成方法不同，這些方法通常需要對靜態分辨率圖像進行複雜的後處理操作，ResAdapter直接生成具有動態分辨率的圖像。特別是，在深入了解純分辨率先驗知識後，ResAdapter在通用數據集上訓練，使用個性化擴散模型生成無分辨率限制的圖像，同時保留其原始風格領域。全面的實驗表明，ResAdapter僅需0.5M即可處理具有靈活分辨率的圖像，適用於任意擴散模型。更廣泛的實驗表明，ResAdapter與其他模塊（例如ControlNet、IP-Adapter和LCM-LoRA）兼容，可用於跨多種分辨率生成圖像，並可集成到其他多分辨率模型（例如ElasticDiffusion）中，高效生成更高分辨率的圖像。項目鏈接為https://res-adapter.github.io
English
Recent advancement in text-to-image models (e.g., Stable Diffusion) and
corresponding personalized technologies (e.g., DreamBooth and LoRA) enables
individuals to generate high-quality and imaginative images. However, they
often suffer from limitations when generating images with resolutions outside
of their trained domain. To overcome this limitation, we present the Resolution
Adapter (ResAdapter), a domain-consistent adapter designed for diffusion models
to generate images with unrestricted resolutions and aspect ratios. Unlike
other multi-resolution generation methods that process images of static
resolution with complex post-process operations, ResAdapter directly generates
images with the dynamical resolution. Especially, after learning a deep
understanding of pure resolution priors, ResAdapter trained on the general
dataset, generates resolution-free images with personalized diffusion models
while preserving their original style domain. Comprehensive experiments
demonstrate that ResAdapter with only 0.5M can process images with flexible
resolutions for arbitrary diffusion models. More extended experiments
demonstrate that ResAdapter is compatible with other modules (e.g., ControlNet,
IP-Adapter and LCM-LoRA) for image generation across a broad range of
resolutions, and can be integrated into other multi-resolution model (e.g.,
ElasticDiffusion) for efficiently generating higher-resolution images. Project
link is https://res-adapter.github.io