整体遗忘基准：用于文本到图像扩散模型遗忘的多方面评估

Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearning

October 8, 2024

作者: Saemi Moon, Minjong Lee, Sangdon Park, Dongwoo Kim

cs.AI

摘要

随着文本到图像扩散模型的进步足以用于商业应用，人们也越来越担心其潜在的恶意和有害用途。模型遗忘被提出来缓解这些担忧，通过从预训练模型中删除不需要的和潜在有害的信息。到目前为止，遗忘的成功主要通过未遗忘的模型是否能生成目标概念并保持图像质量来衡量。然而，遗忘通常在有限的场景下进行测试，目前文献中对遗忘的副作用几乎没有研究。在这项工作中，我们通过五个关键方面全面分析了在不同场景下的遗忘。我们的研究揭示了每种方法都存在副作用或限制，尤其是在更复杂和现实情况下。通过发布我们的全面评估框架以及源代码和工件，我们希望激发这一领域的进一步研究，从而推动更可靠和有效的遗忘方法的发展。

English

As text-to-image diffusion models become advanced enough for commercial applications, there is also increasing concern about their potential for malicious and harmful use. Model unlearning has been proposed to mitigate the concerns by removing undesired and potentially harmful information from the pre-trained model. So far, the success of unlearning is mainly measured by whether the unlearned model can generate a target concept while maintaining image quality. However, unlearning is typically tested under limited scenarios, and the side effects of unlearning have barely been studied in the current literature. In this work, we thoroughly analyze unlearning under various scenarios with five key aspects. Our investigation reveals that every method has side effects or limitations, especially in more complex and realistic situations. By releasing our comprehensive evaluation framework with the source codes and artifacts, we hope to inspire further research in this area, leading to more reliable and effective unlearning methods.