FedOT：基于水印的联邦潜在扩散模型所有权验证与泄露溯源

摘要

在联邦学习（FL）框架下训练潜在扩散模型（LDM）因兼具LDM的强大生成能力和FL的隐私保护特性而日益受到关注。然而，FL要求将全局模型与多个参与者共享，这可能导致恶意客户端未经授权分发或转售模型。尽管一个直观的解决方案是在FL中采用现有基于VAE的水印技术来保护LDM，但该方法因两个根本挑战而无法有效应对此类威胁：（1）现有方法支持所有权验证，但缺乏将模型泄露追溯至特定恶意客户端的能力；（2）基于VAE的水印较为脆弱，仅需替换解码器即可轻易移除。本文提出FedOT——首个面向联邦LDM的所有权验证与泄漏追溯框架。具体而言，针对第一个挑战，我们设计了一种分块水印：第一部分用于所有权验证，第二部分用于客户端身份识别。此外，为克服第二个挑战并抵御VAE替换攻击，我们引入潜在向量变换（LVT）技术，通过修改VAE的原始潜在分布来强化VAE与U-Net潜在空间之间的关联。这样一来，任何为移除水印而替换VAE的尝试都将导致图像质量显著下降，致使LDM模型无法使用。大量实验表明，FedOT在所有权限验证与可追溯性方面均实现了优越性能。项目主页：https://spyzixuan.github.io/FedOT/。

English

Training Latent Diffusion Models (LDMs) within Federated Learning (FL) has attracted increasing attention due to its ability to combine the powerful generative capacity of LDMs with the privacy-preserving properties of FL. However, FL requires sharing the global model with multiple participants, which risks unauthorized model distribution or resale by malicious clients. While an intuitive approach is to adopt existing VAE-based watermarking techniques for LDMs in FL, this strategy falls short in addressing such threats due to two fundamental challenges: (1) Existing methods support ownership verification but lack the ability to trace model leakage to a specific malicious client; (2) VAE-based watermarks are vulnerable, as they can be removed simply by replacing the decoder with a clean counterpart. In this paper, we propose FedOT, the first framework for ownership verification and leakage tracing in federated LDMs. Specifically, to address the first challenge, we design a chunked watermark, where the first part is for ownership verification, and the second part is used for client identification. Furthermore, to overcome the second challenge and secure the model against VAE replacement attack, we introduce Latent Vector Transformation (LVT), which strengthens the connection between the VAE and U-Net latent spaces by modifying the original latent distribution of the VAE. Consequently, any attempt to replace the VAE for watermark removal leads to significant image quality degradation, making the LDM model unusable. Extensive experiments demonstrate that FedOT achieves superior performance in both ownership verification and traceability. Project page: https://spyzixuan.github.io/FedOT/.