论模加运算的机制与动力学：傅里叶特征、彩票假设与顿悟现象

On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking

February 18, 2026

作者: Jianliang He, Leda Wang, Siyu Chen, Zhuoran Yang

cs.AI

摘要

我们针对双层神经网络如何学习特征以解决模加法任务提出了全面分析。本研究不仅对已训练模型提供了完整的机制性解释，还从理论层面阐明了其训练动力学。尽管已有研究指出单个神经元会学习单频傅里叶特征并进行相位对齐，但尚未完全解释这些特征如何整合为全局解决方案。我们通过形式化训练过程中出现的多样化条件弥补了这一空白——该条件包含相位对称性和频率多样化两个部分，并在过参数化时显现。我们证明这些特性使网络能够协同逼近模加法任务正确逻辑上的缺陷指示函数：单个神经元虽产生噪声信号，但相位对称性实现了多数表决机制以消除噪声，使网络能稳健识别正确和值。此外，我们通过彩票假设机制解释了随机初始化下这些特征的形成机理。梯度流分析表明频率在神经元内部相互竞争，"胜出者"由其初始频谱幅值和相位对齐度决定。从技术角度，我们严格刻画了层级相位耦合动力学，并利用ODE比较引理形式化了竞争格局。最后基于这些发现，我们揭示了顿悟现象的本质，将其描述为包含记忆阶段和两个泛化阶段的三步过程，其驱动力来自损失最小化与权重衰减之间的博弈。

English

We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task. Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics. While prior work has identified that individual neurons learn single-frequency Fourier features and phase alignment, it does not fully explain how these features combine into a global solution. We bridge this gap by formalizing a diversification condition that emerges during training when overparametrized, consisting of two parts: phase symmetry and frequency diversification. We prove that these properties allow the network to collectively approximate a flawed indicator function on the correct logic for the modular addition task. While individual neurons produce noisy signals, the phase symmetry enables a majority-voting scheme that cancels out noise, allowing the network to robustly identify the correct sum. Furthermore, we explain the emergence of these features under random initialization via a lottery ticket mechanism. Our gradient flow analysis proves that frequencies compete within each neuron, with the "winner" determined by its initial spectral magnitude and phase alignment. From a technical standpoint, we provide a rigorous characterization of the layer-wise phase coupling dynamics and formalize the competitive landscape using the ODE comparison lemma. Finally, we use these insights to demystify grokking, characterizing it as a three-stage process involving memorization followed by two generalization phases, driven by the competition between loss minimization and weight decay.