函数注意力：从成对亲和性到函数对应

摘要

学习无限维函数空间之间的映射，即算子学习，对许多机器学习应用至关重要。尽管基于Transformer的算子方法很流行，但它们通常依赖于逐token注意力机制。这些方法将连续场视为离散token，并往往忽略全局函数结构。我们提出函数注意力（Functional Attention），将注意力重新解释为自适应基之间的函数对应。受几何函数映射启发，我们的方法用结构化线性算子替代softmax亲和度，从而获得一种紧凑、可泛化、分辨率无关的表示，能够显式捕捉全局依赖关系。实验表明，函数注意力在许多算子学习任务中（包括求解偏微分方程、3D分割和回归）能达到与最先进方法相当的性能，同时保持对不同离散化的鲁棒性。项目页面详见 https://github.com/xjffff/FUNCATTN。

English

Learning mappings between infinite-dimensional function spaces, or operator learning, is essential for many machine learning applications. Although transformer-based operators are popular, they often rely on token-wise attention. These methods treat continuous fields as discrete tokens and usually ignore the global functional structure. We introduce Functional Attention, which reinterprets attention as a functional correspondence between adaptive bases. Inspired by geometric functional maps, our method replaces softmax affinities with structured linear operators. This yields a compact, generalizable, resolution-invariant representation that explicitly captures global dependencies. Experiments demonstrate that Functional Attention can match state-of-the-art performance in many operator learning tasks, including solving PDEs, 3D segmentation, and regression, while remaining robust to varying discretizations. Project page is available at https://github.com/xjffff/FUNCATTN.