函數注意力：從成對親和性到函數對應

摘要

學習無限維函數空間之間的映射（即算子學習）對許多機器學習應用至關重要。儘管基於Transformer的算子廣受歡迎，但它們通常依賴於逐token的注意力機制。這類方法將連續場視為離散token，往往忽略全局函數結構。我們提出了函數注意力（Functional Attention），將注意力重新解釋為自適應基底之間的函數對應。受幾何函數映射啟發，我們的方法以結構化線性算子取代softmax親和力，從而得到一種緊湊、可泛化且解析度不變的表徵，能明確捕捉全局依賴關係。實驗表明，函數注意力在多項算子學習任務（包括求解PDE、3D分割與回歸）中能達到最先進的性能，同時對不同離散化方式保持穩健。專案頁面位於 https://github.com/xjffff/FUNCATTN。

English

Learning mappings between infinite-dimensional function spaces, or operator learning, is essential for many machine learning applications. Although transformer-based operators are popular, they often rely on token-wise attention. These methods treat continuous fields as discrete tokens and usually ignore the global functional structure. We introduce Functional Attention, which reinterprets attention as a functional correspondence between adaptive bases. Inspired by geometric functional maps, our method replaces softmax affinities with structured linear operators. This yields a compact, generalizable, resolution-invariant representation that explicitly captures global dependencies. Experiments demonstrate that Functional Attention can match state-of-the-art performance in many operator learning tasks, including solving PDEs, 3D segmentation, and regression, while remaining robust to varying discretizations. Project page is available at https://github.com/xjffff/FUNCATTN.