鏈式思考標記是計算機程序中的變量。

Chain-of-Thought Tokens are Computer Program Variables

May 8, 2025

作者: Fangwei Zhu, Peiyi Wang, Zhifang Sui

cs.AI

摘要

思維鏈（Chain-of-Thoughts, CoT）要求大型語言模型（LLMs）在得出最終答案之前生成中間步驟，並已被證明能有效幫助LLMs解決複雜的推理任務。然而，CoT的內在機制在很大程度上仍不明確。本文中，我們通過實證研究探討了CoT標記在LLMs中於兩個組合任務——多位數乘法和動態規劃——上的作用。雖然CoT對於解決這些問題至關重要，但我們發現僅保留存儲中間結果的標記即可達到相當的性能。此外，我們觀察到將中間結果以另一種潛在形式存儲並不會影響模型性能。我們還隨機干預了CoT中的某些值，並注意到後續的CoT標記和最終答案會相應地發生變化。這些發現表明，CoT標記可能像計算機程序中的變量一樣運作，但也存在諸如無意中的捷徑和標記間計算複雜度限制等潛在缺點。代碼和數據可在https://github.com/solitaryzero/CoTs_are_Variables獲取。

English

Chain-of-thoughts (CoT) requires large language models (LLMs) to generate intermediate steps before reaching the final answer, and has been proven effective to help LLMs solve complex reasoning tasks. However, the inner mechanism of CoT still remains largely unclear. In this paper, we empirically study the role of CoT tokens in LLMs on two compositional tasks: multi-digit multiplication and dynamic programming. While CoT is essential for solving these problems, we find that preserving only tokens that store intermediate results would achieve comparable performance. Furthermore, we observe that storing intermediate results in an alternative latent form will not affect model performance. We also randomly intervene some values in CoT, and notice that subsequent CoT tokens and the final answer would change correspondingly. These findings suggest that CoT tokens may function like variables in computer programs but with potential drawbacks like unintended shortcuts and computational complexity limits between tokens. The code and data are available at https://github.com/solitaryzero/CoTs_are_Variables.