擴展本質上可解釋的語言模型

摘要

可解釋性常被視為能力的額外負擔：語言模型通常被訓練為不透明系統，事後再以可靠性難以確立的方法加以解釋。在本研究中，我們挑戰這個前提。我們不採逆向工程模型的方式，而是將可解釋性設為訓練流程中的一項約束，並與語言建模目標一同優化。在橫跨三個數量級的算力規模下，無論是自迴歸還是擴散語言模型，可解釋性都與能力一同提升，而非彼此牴觸。令人意外的是，隨著規模擴大，模型表徵變得更加解耦，也更能與人類可理解的概念對齊。我們以 Steerling-8B 實例化了這個訓練時配方；它是一個具因果注意力遮罩的擴散語言模型。對於任何一組生成的 token，Steerling-8B 都能將輸出歸因於相關的輸入 token、人類可理解的概念，以及訓練資料。這使得閉迴路干預得以實現：透過概念或特徵歸因診斷輸出、檢索相似的訓練資料，並在不重新訓練的情況下，透過概念引導修正行為。Steerling-8B 仍能與以兩倍至十六倍以上算力訓練的開放同級模型競爭，這暗示了一種不同的規模化範式：可解釋性可以被設計進訓練之中，並隨規模擴大而改善。

English

Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.