基於影響力近似的高效機器遺忘

Efficient Machine Unlearning via Influence Approximation

July 31, 2025

作者: Jiawei Liu, Chenwang Wu, Defu Lian, Enhong Chen

cs.AI

摘要

隨著隱私問題日益受到關注，機器遺忘（machine unlearning）技術，旨在使機器學習模型能夠「忘記」特定的訓練數據，已逐漸成為研究焦點。在現有方法中，基於影響力的遺忘方法因其能夠在不重新訓練的情況下估計單個訓練樣本對模型參數的影響而成為主流。然而，這種方法由於需要計算所有訓練樣本和參數的Hessian矩陣及其逆矩陣，導致計算開銷過大，使其在大規模模型和頻繁數據刪除請求的場景中難以實際應用。這凸顯了遺忘的困難性。受認知科學啟發，該領域研究表明記憶比遺忘更容易，本文建立了記憶（增量學習）與遺忘（機器遺忘）之間的理論聯繫。這一聯繫使得機器遺忘問題可以從增量學習的角度來解決。與遺忘（忘記）中耗時的Hessian計算不同，增量學習（記憶）通常依賴於更高效的梯度優化，這支持了上述認知理論。基於這一聯繫，我們從增量學習的視角提出了影響力近似遺忘（Influence Approximation Unlearning, IAU）算法，用於高效的機器遺忘。大量實證評估表明，IAU在移除保證、遺忘效率和模型效用之間實現了優越的平衡，同時在多樣化的數據集和模型架構上超越了現有的最先進方法。我們的代碼可在https://github.com/Lolo1222/IAU獲取。

English

Due to growing privacy concerns, machine unlearning, which aims at enabling machine learning models to ``forget" specific training data, has received increasing attention. Among existing methods, influence-based unlearning has emerged as a prominent approach due to its ability to estimate the impact of individual training samples on model parameters without retraining. However, this approach suffers from prohibitive computational overhead arising from the necessity to compute the Hessian matrix and its inverse across all training samples and parameters, rendering it impractical for large-scale models and scenarios involving frequent data deletion requests. This highlights the difficulty of forgetting. Inspired by cognitive science, which suggests that memorizing is easier than forgetting, this paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning). This connection allows machine unlearning to be addressed from the perspective of incremental learning. Unlike the time-consuming Hessian computations in unlearning (forgetting), incremental learning (memorizing) typically relies on more efficient gradient optimization, which supports the aforementioned cognitive theory. Based on this connection, we introduce the Influence Approximation Unlearning (IAU) algorithm for efficient machine unlearning from the incremental perspective. Extensive empirical evaluations demonstrate that IAU achieves a superior balance among removal guarantee, unlearning efficiency, and comparable model utility, while outperforming state-of-the-art methods across diverse datasets and model architectures. Our code is available at https://github.com/Lolo1222/IAU.