模块化加法的作用机制与动力学：傅里叶特征、彩票假说与顿悟现象

On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking

February 18, 2026

作者: Jianliang He, Leda Wang, Siyu Chen, Zhuoran Yang

cs.AI

摘要

我们针对双层神经网络如何学习特征以解决模加法任务提出了全面分析。本研究不仅对已学习模型提供了完整的机制性解释，还从理论上阐释了其训练动态。尽管已有研究指出单个神经元会学习单频傅里叶特征并进行相位对齐，但尚未完全阐明这些特征如何整合为全局解决方案。我们通过形式化训练过程中出现的多样化条件（包含相位对称性与频率多样化两个部分）填补了这一空白，并证明这些特性使网络能够通过集体协作近似实现模加法任务的正确逻辑判定函数。虽然单个神经元会产生噪声信号，但相位对称性实现了多数表决机制，通过噪声抵消使网络能稳健识别正确求和结果。此外，我们通过彩票假设机制揭示了随机初始化下这些特征的形成原理。梯度流分析表明频率会在神经元内部竞争，其"胜出者"由初始频谱幅值与相位对齐程度决定。在技术层面，我们严格刻画了层级相位耦合动态，并运用ODE比较引理形式化了竞争格局。最终基于这些发现，我们将顿悟现象阐释为包含记忆阶段和两个泛化阶段的三步过程，该过程由损失最小化与权重衰减之间的竞争所驱动。

English

We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task. Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics. While prior work has identified that individual neurons learn single-frequency Fourier features and phase alignment, it does not fully explain how these features combine into a global solution. We bridge this gap by formalizing a diversification condition that emerges during training when overparametrized, consisting of two parts: phase symmetry and frequency diversification. We prove that these properties allow the network to collectively approximate a flawed indicator function on the correct logic for the modular addition task. While individual neurons produce noisy signals, the phase symmetry enables a majority-voting scheme that cancels out noise, allowing the network to robustly identify the correct sum. Furthermore, we explain the emergence of these features under random initialization via a lottery ticket mechanism. Our gradient flow analysis proves that frequencies compete within each neuron, with the "winner" determined by its initial spectral magnitude and phase alignment. From a technical standpoint, we provide a rigorous characterization of the layer-wise phase coupling dynamics and formalize the competitive landscape using the ODE comparison lemma. Finally, we use these insights to demystify grokking, characterizing it as a three-stage process involving memorization followed by two generalization phases, driven by the competition between loss minimization and weight decay.