KAN：科尔莫戈洛夫-阿诺德网络

KAN: Kolmogorov-Arnold Networks

April 30, 2024

作者: Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, Max Tegmark

cs.AI

摘要

受 Kolmogorov-Arnold 表示定理启发，我们提出 Kolmogorov-Arnold 网络（KANs）作为多层感知器（MLPs）的有希望的替代方案。虽然 MLPs 在节点（“神经元”）上具有固定的激活函数，但 KANs 在边缘（“权重”）上具有可学习的激活函数。KANs 根本没有线性权重 -- 每个权重参数都被参数化为样条函数的单变量函数所取代。我们展示了这一看似简单的改变使 KANs 在准确性和可解释性方面胜过 MLPs。在准确性方面，较小的 KANs 可以在数据拟合和偏微分方程求解中实现与较大的 MLPs 相当或更好的准确性。从理论和经验上看，KANs 具有比 MLPs 更快的神经缩放规律。在可解释性方面，KANs 可以直观可视化，并且可以轻松地与人类用户交互。通过数学和物理学中的两个示例，展示了 KANs 是有用的合作者，帮助科学家（重新）发现数学和物理定律。总之，KANs 是 MLPs 的有希望的替代方案，为进一步改进今天严重依赖于 MLPs 的深度学习模型打开了机会。

English

Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today's deep learning models which rely heavily on MLPs.