通用推理模型

摘要

通用變換器（UT）在複雜推理任務（如ARC-AGI和數獨求解）中已獲廣泛應用，然其性能提升的具體來源仍待深入探究。本研究系統性分析多種UT變體，發現ARC-AGI任務的性能改善主要源於變換器的循環歸納偏置與強大非線性組件，而非精細的架構設計。基於此發現，我們提出通用推理模型（URM），通過引入短卷積與截斷反向傳播機制增強UT架構。該方法顯著提升推理性能，在ARC-AGI 1和ARC-AGI 2數據集上分別達到53.8%和16.0%的pass@1最佳成績。程式碼已開源於：https://github.com/zitian-gao/URM。

English

Universal transformers (UTs) have been widely used for complex reasoning tasks such as ARC-AGI and Sudoku, yet the specific sources of their performance gains remain underexplored. In this work, we systematically analyze UTs variants and show that improvements on ARC-AGI primarily arise from the recurrent inductive bias and strong nonlinear components of Transformer, rather than from elaborate architectural designs. Motivated by this finding, we propose the Universal Reasoning Model (URM), which enhances the UT with short convolution and truncated backpropagation. Our approach substantially improves reasoning performance, achieving state-of-the-art 53.8% pass@1 on ARC-AGI 1 and 16.0% pass@1 on ARC-AGI 2. Our code is avaliable at https://github.com/zitian-gao/URM.