探索視覺變壓器中影響力神經元路徑

Discovering Influential Neuron Path in Vision Transformers

March 12, 2025

作者: Yifan Wang, Yifei Liu, Yingdong Shi, Changming Li, Anqi Pang, Sibei Yang, Jingyi Yu, Kan Ren

cs.AI

摘要

視覺Transformer模型展現出強大的能力，卻仍難以被人類理解，這為實際應用帶來了挑戰與風險。儘管先前的研究嘗試通過輸入歸因和神經元角色分析來揭示這些模型的神秘面紗，但在考慮層級信息及跨層信息流動的整體路徑方面存在顯著空白。本文中，我們探討了視覺Transformer內部影響力神經元路徑的重要性，這是一條從模型輸入到輸出、對模型推理影響最為顯著的神經元路徑。我們首先提出了一種聯合影響度量方法，用於評估一組神經元對模型結果的貢獻。進一步地，我們提供了一種層級漸進的神經元定位方法，該方法高效地選取每一層中最具影響力的神經元，旨在發現目標模型內從輸入到輸出的關鍵神經元路徑。實驗結果表明，我們的方法在發現信息流動的最具影響力神經元路徑方面，優於現有的基準解決方案。此外，這些神經元路徑揭示了視覺Transformer在處理同一圖像類別內的視覺信息時，展現出特定的內部工作機制。我們進一步分析了這些神經元在圖像分類任務中的關鍵作用，展示出所發現的神經元路徑已保留了模型在下游任務上的能力，這也可能為模型剪枝等實際應用提供啟示。包含實現代碼的項目網站可訪問：https://foundation-model-research.github.io/NeuronPath/。

English

Vision Transformer models exhibit immense power yet remain opaque to human understanding, posing challenges and risks for practical applications. While prior research has attempted to demystify these models through input attribution and neuron role analysis, there's been a notable gap in considering layer-level information and the holistic path of information flow across layers. In this paper, we investigate the significance of influential neuron paths within vision Transformers, which is a path of neurons from the model input to output that impacts the model inference most significantly. We first propose a joint influence measure to assess the contribution of a set of neurons to the model outcome. And we further provide a layer-progressive neuron locating approach that efficiently selects the most influential neuron at each layer trying to discover the crucial neuron path from input to output within the target model. Our experiments demonstrate the superiority of our method finding the most influential neuron path along which the information flows, over the existing baseline solutions. Additionally, the neuron paths have illustrated that vision Transformers exhibit some specific inner working mechanism for processing the visual information within the same image category. We further analyze the key effects of these neurons on the image classification task, showcasing that the found neuron paths have already preserved the model capability on downstream tasks, which may also shed some lights on real-world applications like model pruning. The project website including implementation code is available at https://foundation-model-research.github.io/NeuronPath/.