通过影响力近似实现高效机器遗忘

Efficient Machine Unlearning via Influence Approximation

July 31, 2025

作者: Jiawei Liu, Chenwang Wu, Defu Lian, Enhong Chen

cs.AI

摘要

随着隐私问题日益受到关注，旨在使机器学习模型能够“遗忘”特定训练数据的机器遗忘技术，正获得越来越多的重视。在现有方法中，基于影响力的遗忘因其无需重新训练即可估算单个训练样本对模型参数影响的能力，已成为一种重要方法。然而，该方法因需计算所有训练样本和参数上的海森矩阵及其逆矩阵，导致计算开销巨大，使其在大规模模型及频繁数据删除请求的场景中难以实际应用，这凸显了遗忘的难度。受认知科学中“记忆比遗忘更容易”这一观点的启发，本文建立了记忆（增量学习）与遗忘（机器遗忘）之间的理论联系。这一联系使得机器遗忘问题可以从增量学习的角度加以解决。与遗忘（机器遗忘）中耗时的海森矩阵计算不同，记忆（增量学习）通常依赖于更为高效的梯度优化，这支持了上述认知理论。基于这一联系，我们提出了从增量视角出发的高效机器遗忘算法——影响力近似遗忘（IAU）。大量实证评估表明，IAU在删除保证、遗忘效率与模型性能可比性之间实现了优越的平衡，并在多种数据集和模型架构上超越了现有最先进方法。我们的代码已发布于https://github.com/Lolo1222/IAU。

English

Due to growing privacy concerns, machine unlearning, which aims at enabling machine learning models to ``forget" specific training data, has received increasing attention. Among existing methods, influence-based unlearning has emerged as a prominent approach due to its ability to estimate the impact of individual training samples on model parameters without retraining. However, this approach suffers from prohibitive computational overhead arising from the necessity to compute the Hessian matrix and its inverse across all training samples and parameters, rendering it impractical for large-scale models and scenarios involving frequent data deletion requests. This highlights the difficulty of forgetting. Inspired by cognitive science, which suggests that memorizing is easier than forgetting, this paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning). This connection allows machine unlearning to be addressed from the perspective of incremental learning. Unlike the time-consuming Hessian computations in unlearning (forgetting), incremental learning (memorizing) typically relies on more efficient gradient optimization, which supports the aforementioned cognitive theory. Based on this connection, we introduce the Influence Approximation Unlearning (IAU) algorithm for efficient machine unlearning from the incremental perspective. Extensive empirical evaluations demonstrate that IAU achieves a superior balance among removal guarantee, unlearning efficiency, and comparable model utility, while outperforming state-of-the-art methods across diverse datasets and model architectures. Our code is available at https://github.com/Lolo1222/IAU.