Pion：一種通過正交等價變換的頻譜保持優化器

Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation

May 12, 2026

作者: Kexuan Shi, Hanxuan Li, Zeju Qiu, Yandong Wen, Simon Buchholz, Weiyang Liu

cs.AI

摘要

我們介紹 Pion，一種基於正交等價變換的保譜優化器，專為大型語言模型（LLM）訓練而設計。與 Adam 和 Muon 等加法優化器不同，Pion 通過左右正交變換更新每個權重矩陣，在訓練過程中保持其奇異值不變。這產生了一種優化機制，能夠調節權重矩陣的幾何結構，同時固定其譜範數。我們推導出 Pion 的更新規則，系統性地檢驗其設計選擇，並分析其收斂行為及若干關鍵性質。實驗結果顯示，Pion 在 LLM 的預訓練與微調中，均能提供穩定且具競爭力的替代方案，相較於標準優化器表現優異。

English

We introduce Pion, a spectrum-preserving optimizer for large language model (LLM) training based on orthogonal equivalence transformation. Unlike additive optimizers such as Adam and Muon, Pion updates each weight matrix through left and right orthogonal transformations, preserving its singular values throughout training. This yields an optimization mechanism that modulates the geometry of weight matrices while keeping their spectral norm fixed. We derive the Pion update rule, systematically examine its design choices, and analyze its convergence behavior along with several key properties. Empirical results show that Pion offers a stable and competitive alternative to standard optimizers for both LLM pretraining and finetuning.