FedOT: 基于水印的联邦潜扩散模型所有权验证与泄露追踪

摘要

在聯邦學習（FL）框架中訓練潛在擴散模型（LDM）日益受到關注，因為它能將LDM強大的生成能力與FL的隱私保護特性相結合。然而，FL需要將全局模型共享給多個參與者，這可能導致惡意客戶端未經授權分發或轉售模型。雖然直觀的做法是將現有的基於VAE的浮水印技術應用於FL中的LDM，但由於兩個根本性挑戰，該策略在應對此類威脅時存在不足：（1）現有方法支援所有權驗證，但缺乏將模型洩漏追溯到特定惡意客戶端的能力；（2）基於VAE的浮水印較脆弱，因為只需用乾淨的解碼器替換即可輕易移除。本文提出FedOT，首個針對聯邦LDM的所有權驗證與洩漏追溯框架。具體而言，為解決第一個挑戰，我們設計了一種分塊浮水印，第一部分用於所有權驗證，第二部分用於客戶端識別。此外，為克服第二個挑戰並保護模型免受VAE替換攻擊，我們引入潛在向量轉換（LVT），通過修改VAE原始潛在分佈來強化VAE與U-Net潛在空間之間的關聯。因此，任何試圖替換VAE以移除浮水印的行為都會導致影像品質顯著下降，使LDM模型無法使用。大量實驗證明，FedOT在所有權驗證與可追溯性方面均達到優異性能。專案頁面：https://spyzixuan.github.io/FedOT/。

English

Training Latent Diffusion Models (LDMs) within Federated Learning (FL) has attracted increasing attention due to its ability to combine the powerful generative capacity of LDMs with the privacy-preserving properties of FL. However, FL requires sharing the global model with multiple participants, which risks unauthorized model distribution or resale by malicious clients. While an intuitive approach is to adopt existing VAE-based watermarking techniques for LDMs in FL, this strategy falls short in addressing such threats due to two fundamental challenges: (1) Existing methods support ownership verification but lack the ability to trace model leakage to a specific malicious client; (2) VAE-based watermarks are vulnerable, as they can be removed simply by replacing the decoder with a clean counterpart. In this paper, we propose FedOT, the first framework for ownership verification and leakage tracing in federated LDMs. Specifically, to address the first challenge, we design a chunked watermark, where the first part is for ownership verification, and the second part is used for client identification. Furthermore, to overcome the second challenge and secure the model against VAE replacement attack, we introduce Latent Vector Transformation (LVT), which strengthens the connection between the VAE and U-Net latent spaces by modifying the original latent distribution of the VAE. Consequently, any attempt to replace the VAE for watermark removal leads to significant image quality degradation, making the LDM model unusable. Extensive experiments demonstrate that FedOT achieves superior performance in both ownership verification and traceability. Project page: https://spyzixuan.github.io/FedOT/.