加法的形狀：大型語言模型中算術的幾何結構

摘要

大型語言模型在基本算術中表現出看似矛盾的脆弱性，暗示內部計算與離散輸出之間存在脫節。通過分析多運算元加法過程中的殘差流幾何，我們辨識出「等原始和軌跡」（Iso-Raw-Sum Trajectory, IRST）——一種由語義數字錨定、並受連續進位纖維調製的幾何結構。我們提出「噪聲量化模型」來解釋此幾何現象，將算術錯誤歸因於「幾何滑移」（Geometric Slippages），即內部神經噪聲推動連續的潛在進位勢跨越量化閾值所致。此幾何框架進一步闡明了「探針通用性」（Probe Versatility），解釋了輕量級探針如何從單一激活向量中分離出共存潛在訊號（例如真實答案與幻覺）。最後，我們通過一種幾何一致性檢查方法驗證了這些見解，該方法能在推論過程中有效檢測並修正此類量化失效。我們的程式碼已公開於 https://github.com/RL-MIND/Shape-of-Addition。

English

Large Language Models exhibit paradoxical fragility in fundamental arithmetic, implying a disconnect between internal computation and discrete output. By analyzing the residual stream geometry during multi-operand addition, we identify the Iso-Raw-Sum Trajectory (IRST), a geometric structure where representations are anchored by semantic digits and modulated by continuous carry fibers. We propose the Noisy Quantization Model to explain this geometry, framing arithmetic errors as Geometric Slippages caused by internal neural noise pushing a continuous, latent Carry Potential across quantization thresholds. This geometric framework further elucidates Probe Versatility, explaining how lightweight probes can disentangle coexisting latent signals (such as ground truth versus hallucination) from a single activation vector. Finally, we validate these insights through a geometric consistency check method that effectively detects and corrects these quantization failures during inference. Our code is available at https://github.com/RL-MIND/Shape-of-Addition.