損失函數看不到基底，但 Adam 看得到。

摘要

在因子分解模型 W = UV^⊤ 上，梯度下降隱式偏向低秩解，而 Adam 從相同的小初始化開始則不然。我們將此差異追溯至損失的規範對稱性，即其在 (U, V) ↦ (UQ, VQ) 下的不變性。梯度流的低秩機制僅在優化器具備規範等變性時才可供該優化器使用；此條件是遷移的必要條件，但不足以保證低秩恢復。梯度下降、動量、共享標量 Adam、Muon 和 Shampoo 滿足該條件；Adam、RMSProp 及其他逐坐標方法則不滿足。一個結構定理將無記憶等變規則刻畫為恰好是由 Gram 矩陣決定的左預條件子，一個遷移定理將梯度流的路徑性質傳遞給共享標量流。接著，我們在欠定矩陣感知中按相對於植入真實值的恢復誤差，對九種更新規則進行排序。從逐坐標到共享標量預條件處理的單參數族單調地恢復了該偏差，從而將各向異性隔離為原因。一個「頻譜調度」調和了關於 Muon 的兩份相互矛盾的報告：等速率更新能精確恢復低秩目標，但隨著頻譜尾部的增長而失去優勢。在變壓器中，Adam 在第一步就將兩個規範等價的初始化區分開來，而等變優化器在浮點精度內仍保持相同；最終，逐頭不變量 W_Q^⊤ W_K 的相對 Frobenius 距離相差 56%，這是任何逐頭旋轉都無法彌合的差距。在兩個高光譜數據集上，於匹配的訓練損失下，梯度下降在最低採樣密度時將留出誤差降低了 43–44%，並達到較低的有效秩。因此，基的選擇不是調參細節，而是關於優化器選取哪個插值模型的決策。

English

Gradient descent on a factored model W = UV^top is implicitly biased toward low-rank solutions, while Adam, starting from the same small initialization, is not. We trace the difference to the gauge symmetry of the loss, its invariance under (U, V) mapsto (UQ, VQ). Gradient flow's low-rank mechanism is available to an optimizer only if that optimizer is gauge-equivariant, a condition necessary for the transfer but not sufficient for low-rank recovery. Gradient descent, momentum, "shared-scalar" Adam, Muon, and Shampoo satisfy it. Adam, RMSProp, and the other coordinate-wise methods do not. A structure theorem characterizes the memoryless equivariant rules as exactly the Gram-determined left preconditioners, and a transfer theorem carries gradient flow's pathwise properties to common-scalar flows. We then sort nine update rules on underdetermined matrix sensing by recovery error against the planted ground truth. A one-parameter family from coordinate-wise to shared-scalar preconditioning restores the bias monotonically, isolating anisotropy as the cause. A "spectral schedule" reconciles two opposing reports about Muon: equal-rate updates recover exactly low-rank targets but lose their edge as the spectral tail grows. In transformers, Adam separates two gauge-equivalent initializations at the first step, where the equivariant optimizers stay at float precision, and ends with the per-head invariants W_Q^top W_K 56% apart in relative Frobenius distance, a gap no per-head rotation can close. On two hyperspectral datasets at matched training loss, gradient descent cuts held-out error by 43-44% at the lowest sampling density, and at lower effective rank. Basis choice is therefore not a tuning detail but a decision about which interpolant the optimizer selects.