不变图转换器

Invariant Graph Transformer

December 13, 2023

作者: Zhe Xu, Menghai Pan, Yuzhong Chen, Huiyuan Chen, Yuchen Yan, Mahashweta Das, Hanghang Tong

cs.AI

摘要

理性发现被定义为找到输入数据的一个子集，最大程度地支持下游任务的预测。在图机器学习背景下，图理性被定义为定位给定图拓扑中的关键子图，这基本上决定了预测结果。与理性子图相反，其余子图被称为环境子图。图理性化可以增强模型性能，因为图理性和预测标签之间的映射被视为不变的，根据假设。为了确保提取的理性子图具有区分能力，应用了一种名为“干预”的关键技术。干预的核心思想是，鉴于任何变化的环境子图，来自理性子图的语义是不变的，这保证了正确的预测结果。然而，现有的几乎所有关于图数据的理性化工作都在图级别上开发其干预策略，这是粗粒度的。在本文中，我们提出了针对图数据的精心设计的干预策略。我们的想法受到Transformer模型的发展驱动，其自注意力模块提供了输入节点之间丰富的交互。基于自注意力模块，我们提出的不变图Transformer（IGT）可以实现细粒度，更具体地说，节点级和虚拟节点级的干预。我们的全面实验涉及7个真实世界的数据集，提出的IGT相对于13种基准方法显示出显著的性能优势。

English

Rationale discovery is defined as finding a subset of the input data that maximally supports the prediction of downstream tasks. In graph machine learning context, graph rationale is defined to locate the critical subgraph in the given graph topology, which fundamentally determines the prediction results. In contrast to the rationale subgraph, the remaining subgraph is named the environment subgraph. Graph rationalization can enhance the model performance as the mapping between the graph rationale and prediction label is viewed as invariant, by assumption. To ensure the discriminative power of the extracted rationale subgraphs, a key technique named "intervention" is applied. The core idea of intervention is that given any changing environment subgraphs, the semantics from the rationale subgraph is invariant, which guarantees the correct prediction result. However, most, if not all, of the existing rationalization works on graph data develop their intervention strategies on the graph level, which is coarse-grained. In this paper, we propose well-tailored intervention strategies on graph data. Our idea is driven by the development of Transformer models, whose self-attention module provides rich interactions between input nodes. Based on the self-attention module, our proposed invariant graph Transformer (IGT) can achieve fine-grained, more specifically, node-level and virtual node-level intervention. Our comprehensive experiments involve 7 real-world datasets, and the proposed IGT shows significant performance advantages compared to 13 baseline methods.