不變圖轉換器

Invariant Graph Transformer

December 13, 2023

作者: Zhe Xu, Menghai Pan, Yuzhong Chen, Huiyuan Chen, Yuchen Yan, Mahashweta Das, Hanghang Tong

cs.AI

摘要

理性發現被定義為尋找輸入數據的子集，該子集最大程度地支持下游任務的預測。在圖機器學習背景下，圖理性被定義為定位給定圖拓撲中的關鍵子圖，這基本上決定了預測結果。與理性子圖相反，其餘子圖被稱為環境子圖。圖理性化可以提高模型性能，因為假設圖理性與預測標籤之間的映射是不變的。為了確保提取的理性子圖具有區分能力，應用了一種名為“干預”的關鍵技術。干預的核心思想是，對於任何變化的環境子圖，來自理性子圖的語義是不變的，這保證了正確的預測結果。然而，現有的幾乎所有圖數據的理性化工作都是在圖級別上開發其干預策略，這是粗粒度的。在本文中，我們提出了針對圖數據的精心設計的干預策略。我們的想法受到Transformer模型的發展驅動，其自注意模塊提供了輸入節點之間豐富的交互作用。基於自注意模塊，我們提出的不變圖Transformer（IGT）可以實現精細粒度的，更具體地說，節點級和虛擬節點級的干預。我們的全面實驗涉及7個真實世界的數據集，提出的IGT相對於13個基準方法表現出顯著的性能優勢。

English

Rationale discovery is defined as finding a subset of the input data that maximally supports the prediction of downstream tasks. In graph machine learning context, graph rationale is defined to locate the critical subgraph in the given graph topology, which fundamentally determines the prediction results. In contrast to the rationale subgraph, the remaining subgraph is named the environment subgraph. Graph rationalization can enhance the model performance as the mapping between the graph rationale and prediction label is viewed as invariant, by assumption. To ensure the discriminative power of the extracted rationale subgraphs, a key technique named "intervention" is applied. The core idea of intervention is that given any changing environment subgraphs, the semantics from the rationale subgraph is invariant, which guarantees the correct prediction result. However, most, if not all, of the existing rationalization works on graph data develop their intervention strategies on the graph level, which is coarse-grained. In this paper, we propose well-tailored intervention strategies on graph data. Our idea is driven by the development of Transformer models, whose self-attention module provides rich interactions between input nodes. Based on the self-attention module, our proposed invariant graph Transformer (IGT) can achieve fine-grained, more specifically, node-level and virtual node-level intervention. Our comprehensive experiments involve 7 real-world datasets, and the proposed IGT shows significant performance advantages compared to 13 baseline methods.