TC-LoRA：時序調控條件式LoRA於自適應擴散控制之應用

摘要

当前可控扩散模型通常依赖于固定架构，通过修改中间激活来注入基于新模态的指导。这种方法在动态、多阶段的去噪过程中采用静态条件策略，限制了模型在生成从粗结构到细细节演变时调整其响应的能力。我们引入了TC-LoRA（时间调制条件LoRA），这是一种新范式，通过直接条件化模型权重实现动态、上下文感知的控制。我们的框架利用超网络即时生成LoRA适配器，根据时间和用户条件为冻结的主干网络在每一步扩散过程中定制权重修改。这一机制使模型能够学习并执行一种明确的、自适应的策略，在整个生成过程中应用条件指导。通过在多种数据领域的实验，我们证明与静态的、基于激活的方法相比，这种动态的、参数化的控制显著提高了生成保真度和对空间条件的遵循度。TC-LoRA确立了一种替代方法，其中模型的条件策略通过其权重的更深层次功能适应来修改，使得控制能够与任务和生成阶段的动态需求保持一致。

English

Current controllable diffusion models typically rely on fixed architectures that modify intermediate activations to inject guidance conditioned on a new modality. This approach uses a static conditioning strategy for a dynamic, multi-stage denoising process, limiting the model's ability to adapt its response as the generation evolves from coarse structure to fine detail. We introduce TC-LoRA (Temporally Modulated Conditional LoRA), a new paradigm that enables dynamic, context-aware control by conditioning the model's weights directly. Our framework uses a hypernetwork to generate LoRA adapters on-the-fly, tailoring weight modifications for the frozen backbone at each diffusion step based on time and the user's condition. This mechanism enables the model to learn and execute an explicit, adaptive strategy for applying conditional guidance throughout the entire generation process. Through experiments on various data domains, we demonstrate that this dynamic, parametric control significantly enhances generative fidelity and adherence to spatial conditions compared to static, activation-based methods. TC-LoRA establishes an alternative approach in which the model's conditioning strategy is modified through a deeper functional adaptation of its weights, allowing control to align with the dynamic demands of the task and generative stage.

TC-LoRA：時序調控條件式LoRA於自適應擴散控制之應用

TC-LoRA: Temporally Modulated Conditional LoRA for Adaptive Diffusion Control

摘要

Support