K-LoRA：實現無需訓練即可融合任意主題與風格LoRA的關鍵技術

K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs

February 25, 2025

作者: Ziheng Ouyang, Zhen Li, Qibin Hou

cs.AI

摘要

近期研究探索了結合不同LoRA以共同生成學習到的風格與內容。然而，現有方法要么無法同時有效保留原始主體與風格，要么需要額外的訓練。本文主張，LoRA的內在特性能夠有效引導擴散模型融合學習到的主體與風格。基於此洞察，我們提出了K-LoRA，一種簡單而無需訓練的LoRA融合方法。在每個注意力層中，K-LoRA比較待融合的每個LoRA中的Top-K元素，決定選擇哪個LoRA以實現最佳融合。這種選擇機制確保了在融合過程中保留主體與風格最具代表性的特徵，有效平衡了它們的貢獻。實驗結果表明，所提方法有效整合了原始LoRA學習到的主體與風格信息，在質性與量化結果上均優於現有的基於訓練的方法。

English

Recent studies have explored combining different LoRAs to jointly generate learned style and content. However, existing methods either fail to effectively preserve both the original subject and style simultaneously or require additional training. In this paper, we argue that the intrinsic properties of LoRA can effectively guide diffusion models in merging learned subject and style. Building on this insight, we propose K-LoRA, a simple yet effective training-free LoRA fusion approach. In each attention layer, K-LoRA compares the Top-K elements in each LoRA to be fused, determining which LoRA to select for optimal fusion. This selection mechanism ensures that the most representative features of both subject and style are retained during the fusion process, effectively balancing their contributions. Experimental results demonstrate that the proposed method effectively integrates the subject and style information learned by the original LoRAs, outperforming state-of-the-art training-based approaches in both qualitative and quantitative results.