群組穩健的機器遺忘

摘要

機器遺忘是一種新興範式，旨在從模型中移除特定訓練數據（即遺忘集）的影響，同時保留其對其餘數據（即保留集）的知識。先前的方法假設遺忘數據在所有訓練數據點中均勻分佈。然而，如果待遺忘的數據在某個群體中占主導地位，我們通過實驗證明，該群體的性能會下降，從而導致公平性問題。本研究針對被忽視的非均勻分佈遺忘集問題，提出了一種簡單有效的策略，通過樣本分佈重加權來減輕主導群體中的性能損失，我們稱之為群體魯棒的機器遺忘。此外，我們提出了MIU（基於互信息的機器遺忘），這是首個在近似機器遺忘中實現群體魯棒性的方法。MIU通過最小化模型特徵與群體信息之間的互信息，在實現遺忘的同時減少遺忘集中主導群體的性能下降。此外，MIU利用樣本分佈重加權和與原始模型的互信息校準來保持群體魯棒性。我們在三個數據集上進行了實驗，結果表明MIU優於標準方法，在實現遺忘的同時不損害模型的魯棒性。源代碼可在https://github.com/tdemin16/group-robust_machine_unlearning獲取。

English

Machine unlearning is an emerging paradigm to remove the influence of specific training data (i.e., the forget set) from a model while preserving its knowledge of the rest of the data (i.e., the retain set). Previous approaches assume the forget data to be uniformly distributed from all training datapoints. However, if the data to unlearn is dominant in one group, we empirically show that performance for this group degrades, leading to fairness issues. This work tackles the overlooked problem of non-uniformly distributed forget sets, which we call group-robust machine unlearning, by presenting a simple, effective strategy that mitigates the performance loss in dominant groups via sample distribution reweighting. Moreover, we present MIU (Mutual Information-aware Machine Unlearning), the first approach for group robustness in approximate machine unlearning. MIU minimizes the mutual information between model features and group information, achieving unlearning while reducing performance degradation in the dominant group of the forget set. Additionally, MIU exploits sample distribution reweighting and mutual information calibration with the original model to preserve group robustness. We conduct experiments on three datasets and show that MIU outperforms standard methods, achieving unlearning without compromising model robustness. Source code available at https://github.com/tdemin16/group-robust_machine_unlearning.

群組穩健的機器遺忘

Group-robust Machine Unlearning

摘要

Summary

Support

Support