神經網路可證明學習群體組成的譜表示

摘要

理解神經網路訓練過程中結構化內部結構如何湧現，是深度學習研究的核心問題。我們透過群複合任務來探討此現象：訓練一個兩層神經網路，用以預測有限群 G 中元素的 g₁ ∗ g₂ 結果。藉由將投影梯度流提升至傅立葉域，我們證明訓練動態由表示論能量泛函上的黎曼梯度上升所支配。我們證明，在隨機初始化條件下，該流驅使每個神經元幾乎必然收斂至單一不可約表示，而跨層傅立葉係數則達成旋轉秩－對齊。此框架提供了特徵學習的表示論詮釋，並刻畫了矩陣值群表示中一種新穎的低秩壓縮現象。此外，對於阿貝爾群，我們給出完整的總體層次描述：隨機初始化促進跨非平凡表示的均勻多樣化，並誘導哈爾均勻相位，透過多數投票機制聯合近似指示函數。我們進一步證明，相位對齊與表示競爭均以指數收斂率出現。

English

Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group composition task, where a two-layer neural network is trained to predict g_1 star g_2 for elements of a finite group G. By lifting the projected gradient flow to the Fourier domain, we demonstrate that the training dynamics are governed by a Riemannian gradient ascent on a representation-theoretic energy functional. We prove that, under random initialization, this flow drives each neuron to converge almost surely toward a single irreducible representation, while the cross-layer Fourier coefficients achieve a rotational rank-one alignment. This framework provides a representation-theoretic account of feature learning and characterizes a novel low-rank compression phenomenon for matrix-valued group representations. Moreover, for Abelian groups, we provide a complete population-level description: random initialization promotes uniform diversification across nontrivial representations and induces Haar-uniform phases, jointly approximating the indicator via a majority-vote mechanism. We further prove that both phase alignment and representation competition emerge with exponential convergence rates.