有效推理链降低内在维度

摘要

思维链推理及其变体方法显著提升了语言模型在复杂推理任务中的表现，但不同策略促进泛化能力的具体机制尚未明晰。现有解释多指向测试时计算量的增加或结构化引导，然而要确立这些因素与泛化能力之间的量化关联仍具挑战。本研究提出以内在维度作为量化指标来刻画推理链的有效性——该指标指模型在特定任务上达到给定准确率阈值所需的最小维度数。通过固定模型架构而改变任务表述方式（采用不同推理策略），我们证明有效的推理策略能持续降低任务的内在维度。基于Gemma-3 1B和4B模型在GSM8K数据集上的验证表明，推理策略的内在维度与其在分布内和分布外数据上的泛化性能呈强负相关。这些发现揭示：有效的推理链通过以更少参数实现任务信息的更好压缩来促进学习，从而为分析推理过程提供了新的量化指标。

English

Chain-of-thought (CoT) reasoning and its variants have substantially improved the performance of language models on complex reasoning tasks, yet the precise mechanisms by which different strategies facilitate generalization remain poorly understood. While current explanations often point to increased test-time computation or structural guidance, establishing a consistent, quantifiable link between these factors and generalization remains challenging. In this work, we identify intrinsic dimensionality as a quantitative measure for characterizing the effectiveness of reasoning chains. Intrinsic dimensionality quantifies the minimum number of model dimensions needed to reach a given accuracy threshold on a given task. By keeping the model architecture fixed and varying the task formulation through different reasoning strategies, we demonstrate that effective reasoning strategies consistently reduce the intrinsic dimensionality of the task. Validating this on GSM8K with Gemma-3 1B and 4B, we observe a strong inverse correlation between the intrinsic dimensionality of a reasoning strategy and its generalization performance on both in-distribution and out-of-distribution data. Our findings suggest that effective reasoning chains facilitate learning by better compressing the task using fewer parameters, offering a new quantitative metric for analyzing reasoning processes.

有效推理链降低内在维度

Effective Reasoning Chains Reduce Intrinsic Dimensionality

摘要

Support