概念滑块：LoRA 适配器用于扩散模型中的精确控制

摘要

我们提出了一种方法，用于创建可解释的概念滑块，从扩散模型中实现图像生成属性的精确控制。我们的方法确定了与一个概念对应的低秩参数方向，同时最小化与其他属性的干扰。通过使用一小组提示或示例图像创建滑块，因此可以为文本或视觉概念创建滑块方向。概念滑块是即插即用的：它们可以高效地组合和连续调节，实现对图像生成的精确控制。在与先前的编辑技术进行定量实验比较中，我们的滑块展示出更强的目标编辑效果，并且干扰更少。我们展示了用于天气、年龄、风格和表情的滑块，以及滑块组合。我们展示了滑块如何从StyleGAN转移潜在空间，以直观编辑文本描述困难的视觉概念。我们还发现我们的方法可以帮助解决Stable Diffusion XL中持续存在的质量问题，包括修复物体变形和修复扭曲的手部。我们的代码、数据和训练好的滑块可在https://sliders.baulab.info/ 上获得。

English

We present a method to create interpretable concept sliders that enable precise control over attributes in image generations from diffusion models. Our approach identifies a low-rank parameter direction corresponding to one concept while minimizing interference with other attributes. A slider is created using a small set of prompts or sample images; thus slider directions can be created for either textual or visual concepts. Concept Sliders are plug-and-play: they can be composed efficiently and continuously modulated, enabling precise control over image generation. In quantitative experiments comparing to previous editing techniques, our sliders exhibit stronger targeted edits with lower interference. We showcase sliders for weather, age, styles, and expressions, as well as slider compositions. We show how sliders can transfer latents from StyleGAN for intuitive editing of visual concepts for which textual description is difficult. We also find that our method can help address persistent quality issues in Stable Diffusion XL including repair of object deformations and fixing distorted hands. Our code, data, and trained sliders are available at https://sliders.baulab.info/

概念滑块：LoRA 适配器用于扩散模型中的精确控制

Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models

摘要

Support