訓練留下痕跡：用於語言模型譜系驗證的中心化殘差特徵

摘要

開放權重的語言模型經常被微調、量化、剪枝與合併，但其來源往往缺乏文件記錄。我們研究無資料白箱血統驗證：權重本身能否揭示兩個相容的模型檢查點是否共享祖先？殘差訓練會在分支產物中產生共享的恆等對齊分量，因此單憑此結構無法確立祖先關係。我們移除該分量並比較各殘差區塊中檢查點特有的結構，產出一個對獨立檢查點校準的對稱血統分數。在殘差MLP與GPT-2基準上，該分數能將微調、LoRA合併、剪枝與量化後的後代模型與獨立模型及蒸餾模型區分開來（AUROC=1.0），從而區分權重祖先關係與行為相似性。在保留函數的檢查點洗白實驗中，權重空間基線喪失區分度或完全失效；我們的分數保持不變，且在GPT-2上的運算速度比最穩健的近鄰基線快76倍。投影配對訊號出現在六個語言模型家族及其他模型中，一項案例研究正確識別了3個相關與7個無關的LLaMA-2公開檢查點。整體而言，這些結果為相容的開放權重語言模型檢查點建立了被動、無資料的來源追溯訊號。

English

Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints