SAEScientist-Bench：AI 代理能否自主進行 SAE 可解釋性研究？

摘要

儘管遞迴自我改進（RSI）的研究主要聚焦於自動化模型訓練流程，可靠的自主開發仍缺乏一項關鍵支柱：事後監測與審計，以理解模型所學並確保安全對齊。機制可解釋性工具是彌合此缺口所不可或缺的，其中稀疏自編碼器（SAEs）作為基石，透過分離可解釋特徵以進行模型檢查與引導。在本文中，我們提出 SAEScientist-Bench，用以評估 AI 代理能否作為科學家，運用 SAE 工具進行自主機制發現。給定一個目標概念，代理設計對比探針，並在 Gemma-2-9B-IT 的 Gemma Scope 字典中導航超過 131K 個特徵，以發現最佳特徵，並依據以 Neuronpedia 為基礎所策劃的專家參考特徵，在激活排名、對比文本上的概念選擇性以及因果引導等方面進行評估。在 10 種代理配置與 20 項任務中，前沿代理展現出真正的發現能力，並在不同評估維度上領先，但仍遠落後於專家基線；在將目標概念與對比控制項分離方面接近專家水準，但在因果生成引導方面則大幅落後。進一步分析顯示，儘管代理能設計對比以排除虛假候選，它們仍經常誤解實驗測量結果。這些結果確立了實驗性模型理解作為閉環自主 AI 研發的一項可測量能力。我們的程式碼可在 https://github.com/Trae1ounG/SAEScientist 取得。

English

While research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a missing pillar: post-hoc monitoring and auditing to understand what models learn and ensure safe alignment. Mechanistic interpretability tools are essential to bridge this gap, among which Sparse Autoencoders (SAEs) serve as a cornerstone by isolating interpretable features for model inspection and steering. In this paper, we introduce SAEScientist-Bench to evaluate whether AI agents can act as scientists utilizing SAE tools for autonomous mechanistic discovery. Given a target concept, an agent designs contrastive probes and navigates a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT to discover the optimal feature, evaluated against curated expert reference features anchored on Neuronpedia across activation rank, concept selectivity on contrastive texts, and causal steering. Across 10 agent configurations and 20 tasks, frontier agents demonstrate genuine discovery capabilities and lead different evaluation dimensions, but remain well behind the expert baseline, approaching expert levels on separating target concepts from contrastive controls while lagging substantially in causal generation steering. Further analysis reveals that although agents can design contrasts to rule out spurious candidates, they frequently misinterpret experimental measurements. These results establish experimental model understanding as a measurable capability for closed-loop autonomous AI R&D. Our code is available at https://github.com/Trae1ounG/SAEScientist.