群体鲁棒的机器遗忘

摘要

机器遗忘是一种新兴范式，旨在从模型中移除特定训练数据（即遗忘集）的影响，同时保留其对剩余数据（即保留集）的知识。先前的方法假设遗忘数据在所有训练数据点中均匀分布。然而，若需遗忘的数据在某一群体中占据主导地位，我们通过实验证明该群体的性能会下降，从而引发公平性问题。本研究针对这一被忽视的非均匀分布遗忘集问题，提出了群体鲁棒性机器遗忘的概念，并介绍了一种简单有效的策略，通过样本分布重加权来缓解主导群体的性能损失。此外，我们提出了MIU（互信息感知的机器遗忘），这是首个在近似机器遗忘中实现群体鲁棒性的方法。MIU通过最小化模型特征与群体信息之间的互信息，在实现遗忘的同时减少遗忘集中主导群体的性能下降。MIU还利用样本分布重加权和与原模型的互信息校准，以保持群体鲁棒性。我们在三个数据集上进行了实验，结果表明MIU优于标准方法，实现了遗忘且不损害模型鲁棒性。源代码可在https://github.com/tdemin16/group-robust_machine_unlearning获取。

English

Machine unlearning is an emerging paradigm to remove the influence of specific training data (i.e., the forget set) from a model while preserving its knowledge of the rest of the data (i.e., the retain set). Previous approaches assume the forget data to be uniformly distributed from all training datapoints. However, if the data to unlearn is dominant in one group, we empirically show that performance for this group degrades, leading to fairness issues. This work tackles the overlooked problem of non-uniformly distributed forget sets, which we call group-robust machine unlearning, by presenting a simple, effective strategy that mitigates the performance loss in dominant groups via sample distribution reweighting. Moreover, we present MIU (Mutual Information-aware Machine Unlearning), the first approach for group robustness in approximate machine unlearning. MIU minimizes the mutual information between model features and group information, achieving unlearning while reducing performance degradation in the dominant group of the forget set. Additionally, MIU exploits sample distribution reweighting and mutual information calibration with the original model to preserve group robustness. We conduct experiments on three datasets and show that MIU outperforms standard methods, achieving unlearning without compromising model robustness. Source code available at https://github.com/tdemin16/group-robust_machine_unlearning.

群体鲁棒的机器遗忘

Group-robust Machine Unlearning

摘要

Summary

Support

Support