數學推理中思維鏈的形態

摘要

大型語言模型（LLMs）在數學推理基準上表現強勁，然而支撐其推理能力中具有數學意義的技能仍未被充分探索。我們提出 SHAPE，一個透過數學教育領域所發展出的兩個視角來分析思維鏈（Chain-of-Thought, CoT）軌跡的框架：（1）語義空間：模型對問題不斷演變的數學詮釋（例如代數、幾何），以及（2）啟發式策略：模型在這些空間內採取的具體數學行動（例如簡化問題、倒推求解）。我們首先使用 SHAPE 分析多種模型的推理模式。研究結果顯示，模型所採用的數學啟發式策略比傳統 CoT 特徵更能解釋最終答案的正確性。此外，模型若將其推理努力集中於少數語義空間，而非探索許多互不相關的空間，則更可能得出正確解答——此模式與人類行為一致。接著，我們利用 SHAPE 視角評估後訓練是否真正提升數學能力。我們發現強化學習會誘發啟發式策略使用上的模式尋求（mode-seeking）現象。最後，我們透過促進多樣化的啟發式策略來後訓練 LLMs，並證明其在提升準確率方面的有效性。整體而言，SHAPE 提供了一個具理論基礎的診斷框架，用以解碼 LLM 推理，並為後訓練 LLM 以增強數學推理開闢了新途徑。我們模型的程式碼可於 https://github.com/holi-lab/SHAPE-of-CoT 取得。

English

Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their reasoning remain underexplored. We introduce SHAPE, a framework that analyzes Chain-of-Thought (CoT) trajectories through two lenses developed in mathematics education: (1) semantic spaces: the model's evolving mathematical interpretations of a problem (e.g., algebraic, geometric), and (2) heuristics: the specific mathematical actions taken within those spaces (e.g., simplifying the problem, working backward). We first use SHAPE to analyze the reasoning patterns of various models. Our findings reveal that the mathematical heuristics employed by a model better explain final answer correctness than traditional CoT features. Furthermore, models are likely to reach correct solutions by concentrating their reasoning effort within a few semantic spaces rather than exploring many disparate ones -- a pattern consistent with human behavior. Next, we utilize the SHAPE lens to evaluate whether post-training truly enhances mathematical proficiency. We find that reinforcement learning induces mode-seeking in heuristic usage. Lastly, we post-train LLMs by promoting diverse heuristics and demonstrate its effectiveness in improving accuracy. Overall, SHAPE provides a theoretically-grounded diagnostic framework for decoding LLM reasoning and offers a new path toward post-training LLMs for math reasoning. The code for our model is available at https://github.com/holi-lab/SHAPE-of-CoT