用於探索、淨化與模型合併的譜重連

摘要

強化學習已成為大型語言模型的標準後訓練配方，但稠密的全參數更新會產生兩個部署相關的瓶頸：壓抑推理效能（常表現為測試時擴展的過早飽和），以及透過多領域訓練或模型合併整合多重能力時的干擾。我們證明，這些更新中對推理有效的部分大多集中在基礎模型的譜空間，由此提出子空間對齊重接（SAR），這是一種事後編輯方法，能保留此譜核心並移除正交分量。因此，SAR能保留推理增益，並濾除壓抑效能或放大跨領域干擾的殘餘更新方向。在數個模型系列與規模中，SAR使用僅約0.58%的總參數提取出緊湊的推理核心：它能保留超過99%的後訓練效能，在數學推理中改善高k探索，並透過在內部模型上改善七個開放基準中的六個，泛化至智能體式編碼。SAR也能透過釋放受壓抑的編碼能力同時維持數學推理與指令跟從，純化混合領域的訓練更新。此外，它還能實現專家間的模型合併，產生的跨領域泛化超越先前的合併基準，甚至超越最佳單一領域專家。總體而言，SAR顯示從參數幾何中提取推理有效的更新，可作為一種無訓練機制來改善推理與多領域效能。

English

Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.