训练留有痕迹：用于语言模型谱系验证的中心化残差签名

摘要

开放权重语言模型经常被微调、量化、剪枝和合并，但其来源往往缺乏文档记录。我们研究无数据白盒谱系验证问题：仅凭权重能否揭示两个兼容模型检查点是否共享祖先？残差训练会在分支产物中产生共享的身份对齐分量，因此仅凭该结构无法确立谱系关系。我们移除该分量，并比较跨残差块的检查点特有结构，从而得到一种以独立检查点进行校准的对称谱系评分。在残差MLP和GPT-2基准上，该评分能够将微调、LoRA合并、剪枝和量化后代与独立模型及蒸馏模型区分开来（AUROC=1.0），从而区分权重谱系与行为相似性。在保持功能的检查点清洗实验中，权重空间基线方法或裕度下降或完全失效；而我们的评分保持不变，并且在GPT-2上比最接近的稳健基线快76倍。投影配对信号出现在六个语言模型家族及更广范围的模型中；一项案例研究正确识别了3个相关和7个不相关的LLaMA-2公开检查点。综合而言，这些结果为兼容的开放权重语言模型检查点确立了一种被动式、无数据的谱系信号。

English

Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints