链式思维标记是计算机程序中的变量。

Chain-of-Thought Tokens are Computer Program Variables

May 8, 2025

作者: Fangwei Zhu, Peiyi Wang, Zhifang Sui

cs.AI

摘要

思维链（CoT）要求大型语言模型（LLMs）在得出最终答案前生成中间步骤，已被证实能有效帮助LLMs解决复杂推理任务。然而，CoT的内在机制在很大程度上仍不明确。本文中，我们通过实证研究探讨了CoT标记在LLMs中于两项组合任务——多位数乘法与动态规划——上的作用。尽管CoT对于解决这些问题至关重要，但我们发现仅保留存储中间结果的标记即可达到相当的性能。此外，我们观察到以另一种潜在形式存储中间结果不会影响模型表现。我们还随机干预了CoT中的某些值，注意到后续CoT标记及最终答案会相应变化。这些发现表明，CoT标记可能类似于计算机程序中的变量，但存在诸如意外捷径及标记间计算复杂度限制等潜在缺陷。代码与数据可在https://github.com/solitaryzero/CoTs_are_Variables获取。

English

Chain-of-thoughts (CoT) requires large language models (LLMs) to generate intermediate steps before reaching the final answer, and has been proven effective to help LLMs solve complex reasoning tasks. However, the inner mechanism of CoT still remains largely unclear. In this paper, we empirically study the role of CoT tokens in LLMs on two compositional tasks: multi-digit multiplication and dynamic programming. While CoT is essential for solving these problems, we find that preserving only tokens that store intermediate results would achieve comparable performance. Furthermore, we observe that storing intermediate results in an alternative latent form will not affect model performance. We also randomly intervene some values in CoT, and notice that subsequent CoT tokens and the final answer would change correspondingly. These findings suggest that CoT tokens may function like variables in computer programs but with potential drawbacks like unintended shortcuts and computational complexity limits between tokens. The code and data are available at https://github.com/solitaryzero/CoTs_are_Variables.