分割式LLM訓練中的隱私失效：返回的梯度使誘餌失效

摘要

我們提出一個兩節點分割式LLM訓練系統的系統安全個案研究；該系統的隱私評估通過，但留下了一個可觀察通道未經測試。可信本地節點（TLN）將受保護的激活值傳送給不可信雲端節點（UCN），UCN回傳其輸出，而持有私有損失的TLN再回傳輸出梯度。UCN所接收的幀將真實行與誘餌行混在一起，而損失函數忽略誘餌行。誘餌行的梯度恰好為零，因此零值的模式便揭露哪些行是真實的。我們用一套事先固定的協議來量測此現象：先以已知強度注入一個洩漏，以證明量測工具能偵測到訊號；再使用隨機打亂標籤的控制組，以證明工具不會在沒有洩漏時誤報；並在運行前設定閾值。在九個種子下，零值模式在每一幀皆識別出真實行，每次執行的4,096個之中有4,096個被識別。針對幀內容的攻擊，相較於固定猜測基線，約每百個token多恢復一個token（+0.65至+1.50個百分點）；隨機打亂標籤的控制組則未恢復出任何內容。第二組運行在將模型品質維持於預算內的配置上重複了此結果，因此該發現並不限於無人會採用的設定。在兩個資料集上，所有這類運行都通過了前向通道的隱私檢查與品質檢查，然而一旦把回傳的梯度納入檢查，便會在同一檢查中失敗。對梯度的每一行進行裁剪與加噪後，该洩漏被關閉，代價是留出交叉熵增加約0.01 nats。系統並非因此就安全：有五類攻擊從未被量測，包括那些跨訓練步驟累積觀察的攻擊。

English

We present a systems-security case study of a two-node split-LLM training system whose privacy evaluation passed while leaving an observable channel untested. The Trusted Local Node (TLN) sends protected activations to the Untrusted Cloud Node (UCN), the UCN returns its output, and TLN, holding the private loss, returns the output gradient. The frame the UCN receives mixes real rows with decoys, and the loss ignores the decoys. Their gradients are exactly zero, so the pattern of zeros reveals which rows were real. We measure it with a protocol fixed in advance: a leak injected at known strength to prove the instrument can see one, a shuffled-label control to prove it does not report absent leaks, and a threshold set before the runs. Across nine seeds, the zeros identified the real rows on every frame, 4,096 of 4,096 per run. An attack on the frame contents recovered about one extra token per hundred over a constant-guess baseline (+0.65 to +1.50 percentage points); the shuffled controls recovered nothing. A second set of runs repeated this on a configuration that keeps model quality within budget, so the finding is not confined to a setting nobody would deploy. On both datasets, every such run passed the forward-channel privacy check and the quality check, yet failed that same check once the returned gradient was included. Clipping and noising each row of the gradient closed the leak for about 0.01 nats of held-out cross-entropy. The system is not thereby safe: five classes of attack, including those accumulating observations across training steps, were never measured.