Omega-S：大型語言模型微調的功能性韌性指標

摘要

在大型語言模型上使用新資料進行微調，會損害其先前學到的知識。我們提出 Omega-S，這是一個僅由權重矩陣計算的即插即用懲罰項：它不需要先前的任務資料、不需要 Fisher 矩陣，也不需要儲存舊權重的副本。它只需在現有訓練迴圈中加入三行程式碼，且每次迭代的運算成本增加不到 4%。 **保留能力。** 在以 LoRA 微調 Llama-3-8B、從程式碼轉換為散文並以 HumanEval 在十個隨機種子下測量的實驗中，Omega-S 在 10 個種子中的 9 個上，比無正規化保留了更多原有能力（絕對 pass@1 從 0.173 提升至 0.238；單尾符號檢定 p=0.011，Wilcoxon 檢定 p=0.006），保留比率從 62.9% 提升至 84.1%。它也優於經過調校的權重衰減（10/10 種子，p=0.002）及調校後的 EWC（8/10，p=0.014），所有實驗組均在相同的實驗場次中重新測量。 **機制：以實測為據而非主觀宣稱。** Omega-S 在構造上具有拓撲性質，其目標函數由 Tr(A^3) 建構而成，但我們測量了其四個因子中實際產生作用的哪一個，結果發現其中三個並未起作用：它們相對於權重的彈性低於或等於 1e-4，而度變異數項則為 9e-3。就目前的實作而言，此複合項可化約為對節點度數變異數的懲罰，這在方形模組中意味著列向量的大小，在非方形模組中則代表方向對齊。我們之所以報告此事，是因為一個方法若其名稱承諾了一件事，而其梯度做的卻是另一件事，就應該明說。我們也列舉了尚未定論的設計選擇，包括一種對比保留的建構方式，它確實達成了設計目標，但在全部十個種子上都使保留能力變得更差。在完全相同設定、相同隨機種子與相同硬體下重複實驗，保留比率的標準差為 0.104。我們尚未發現其他研究對語言模型的低秩微調量化過此變異性，而這個數值為文獻中所有種子配對比較（包括我們自己的比較）提供了誤差下限。我們提供了程式碼、每個種子的結果，以及完整的負面結果記錄。

English

Fine-tuning a large language model on new data degrades what it previously learned. We present Omega-S, a drop-in penalty computed from the weight matrix alone: it needs no previous-task data, no Fisher matrix and no stored copy of the old weights. It is three lines in an existing training loop and adds under 4% to the cost of a step. Retention. On Llama-3-8B with LoRA, fine-tuned from code to prose and measured by HumanEval over ten seeds, Omega-S retains more of the original capability than no regularisation on 9 of 10 seeds (0.173 -> 0.238 absolute pass@1; sign test one-sided p=0.011, Wilcoxon p=0.006), as a retention ratio, 62.9% -> 84.1%. It also beats tuned weight decay on 10 of 10 seeds (p=0.002) and tuned EWC on 8 of 10 (p=0.014), every arm re-measured in the same session. Mechanism, measured rather than asserted. Omega-S is topological by construction, its objective built from Tr(A^3), but we measured which of its four factors actually moves and three do not: their elasticity with respect to the weights is at or below 1e-4, against 9e-3 for the degree-variance term. As implemented, the composite reduces to a penalty on the variance of node degrees, which means row magnitude in square modules and directional alignment in non-square ones. We report this because a method whose name promises one thing and whose gradient does another should say so. We also enumerate the open design choices, including a contrast-preserving construction that does what it was designed to do and makes retention worse on all ten seeds. Repeating an identical configuration, same seed and same hardware, gives a standard deviation of 0.104 in retention ratio. We have not found this quantified for low-rank fine-tuning of language models, and it bounds every seed-paired comparison in this literature, ours included. Code, per-seed results and the full record of negative results are available.