概念滑桿：LoRA 轉接器用於擴散模型中的精確控制

摘要

我們提出了一種方法來創建可解釋的概念滑塊，從擴散模型的圖像生成中實現對屬性的精確控制。我們的方法識別了與一個概念相對應的低秩參數方向，同時最小化與其他屬性的干擾。通過使用少量提示或示例圖像來創建滑塊；因此，滑塊方向可以為文本或視覺概念創建。概念滑塊是即插即用的：它們可以高效地組合並連續調節，實現對圖像生成的精確控制。在與先前編輯技術進行定量實驗比較時，我們的滑塊展示出更強的有針對性編輯，並具有更低的干擾。我們展示了用於天氣、年齡、風格和表情的滑塊，以及滑塊組合。我們展示了如何使用滑塊從StyleGAN轉移潛在特徵，以直觀地編輯對於文本描述困難的視覺概念。我們還發現我們的方法可以幫助解決Stable Diffusion XL中持續的質量問題，包括修復物體變形和修復扭曲的手部。我們的代碼、數據和訓練有素的滑塊可在https://sliders.baulab.info/獲得。

English

We present a method to create interpretable concept sliders that enable precise control over attributes in image generations from diffusion models. Our approach identifies a low-rank parameter direction corresponding to one concept while minimizing interference with other attributes. A slider is created using a small set of prompts or sample images; thus slider directions can be created for either textual or visual concepts. Concept Sliders are plug-and-play: they can be composed efficiently and continuously modulated, enabling precise control over image generation. In quantitative experiments comparing to previous editing techniques, our sliders exhibit stronger targeted edits with lower interference. We showcase sliders for weather, age, styles, and expressions, as well as slider compositions. We show how sliders can transfer latents from StyleGAN for intuitive editing of visual concepts for which textual description is difficult. We also find that our method can help address persistent quality issues in Stable Diffusion XL including repair of object deformations and fixing distorted hands. Our code, data, and trained sliders are available at https://sliders.baulab.info/

概念滑桿：LoRA 轉接器用於擴散模型中的精確控制

Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models

摘要

Support