循环圈圈！

Loop the Loopies!

July 17, 2026

作者: Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang, Ran Tao, Joey Zhou, Bryan Dai

cs.AI

摘要

我们推出了Loopie，这是迄今为止最强大的循环Transformer。Loopie系列包含两个混合专家（MoE）模型：一个200亿参数模型（其中20亿为活跃参数）和一个60亿参数模型（其中6亿为活跃参数）。循环Transformer长期以来面临一项挑战：当预训练计算量增加N倍时，将参数量扩大N倍通常比将模型循环N次效果更佳。Loopie成功攻克了这一难题。大量消融实验（包括与普通30B-A3B模型的对比）表明，在相同计算预算下，Loopie显著优于普通Transformer基线。我们创新的训练后流程赋予了Loopie强大的推理能力。在2025年国际数学奥林匹克竞赛（IMO）和国际物理奥林匹克竞赛（IPhO）中，Loopie在不借助外部工具的情况下取得了金牌级别的表现。

English

We present Loopie, the most powerful looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6Bparameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. Our novel post-training pipeline equips Loopie with strong reasoning abilities. At the 2025 IMO and IPhO, Loopie achieves gold-medal performance without tools.