差分隐私随机梯度下降的隐私-效用优势保证存在根本性局限
Fundamental Limitations of Favorable Privacy-Utility Guarantees for DP-SGD
January 15, 2026
作者: Murat Bilgehan Ertan, Marten van Dijk
cs.AI
摘要
差分隐私随机梯度下降（DP-SGD）是隐私训练的主流范式，但学界对其在最坏情况对抗性隐私定义下的根本局限仍缺乏深入理解。我们在f-差分隐私框架下分析DP-SGD——该框架通过假设检验权衡曲线来刻画隐私特性，并研究单周期内进行M次梯度更新的混洗采样机制。我们推导出可达权衡曲线的显式次优上界，该结果引出了分离度κ的几何下界（即机制权衡曲线与理想随机猜测线之间的最大距离）。由于较大的分离度意味着显著的对抗性优势，有意义的隐私保护需要较小的κ值。然而我们证明，强制保持较小分离度会对高斯噪声乘数σ施加严格下界，这直接限制了可达效用。具体而言，在标准最坏情况对抗模型下，混洗DP-SGD必须满足
σ≥ 1/√(2ln M) 或 κ≥ 1/8∙[1-1/(4πln M)]，
因此无法同时实现强隐私保护与高效用。尽管该边界随M→∞渐近消失，但收敛速度极慢：即使对于实际应用中常见的更新次数，所需噪声量级仍然显著。我们进一步证明该限制在常数因子范围内同样适用于泊松子采样。实验证实该边界所隐含的噪声水平会导致实际训练场景下的精度显著下降，从而揭示了标准最坏情况对抗假设下DP-SGD的关键瓶颈。
English
Differentially Private Stochastic Gradient Descent (DP-SGD) is the dominant paradigm for private training, but its fundamental limitations under worst-case adversarial privacy definitions remain poorly understood. We analyze DP-SGD in the f-differential privacy framework, which characterizes privacy via hypothesis-testing trade-off curves, and study shuffled sampling over a single epoch with M gradient updates. We derive an explicit suboptimal upper bound on the achievable trade-off curve. This result induces a geometric lower bound on the separation κ which is the maximum distance between the mechanism's trade-off curve and the ideal random-guessing line. Because a large separation implies significant adversarial advantage, meaningful privacy requires small κ. However, we prove that enforcing a small separation imposes a strict lower bound on the Gaussian noise multiplier σ, which directly limits the achievable utility. In particular, under the standard worst-case adversarial model, shuffled DP-SGD must satisfy
σge 1{2ln M} quadorquad κge 1{8}!left(1-1{4πln M}right),
and thus cannot simultaneously achieve strong privacy and high utility. Although this bound vanishes asymptotically as M to infty, the convergence is extremely slow: even for practically relevant numbers of updates the required noise magnitude remains substantial. We further show that the same limitation extends to Poisson subsampling up to constant factors. Our experiments confirm that the noise levels implied by this bound leads to significant accuracy degradation at realistic training settings, thus showing a critical bottleneck in DP-SGD under standard worst-case adversarial assumptions.