实现数学推理的自适应性

摘要

数学推理是衡量大型语言模型（LLMs）智能水平的核心指标。然而，现有LLMs在鲁棒性和泛化能力上存在明显不足。本文将这些缺陷归因于虚假推理，即模型仅依据表面特征生成答案。为应对这一挑战，我们提出了AdaR框架，旨在实现自适应推理，使模型基于问题解决逻辑来产生答案。AdaR通过变换变量值合成逻辑等价的查询，并利用RLVR（强化学习与验证反馈）在这些数据上训练模型，以抑制虚假逻辑，同时促进自适应逻辑的运用。为提高数据质量，我们从原始查询中提取问题解决逻辑，通过代码执行生成相应答案，并进行合理性检验。实验结果表明，AdaR显著提升了模型的鲁棒性和泛化能力，在数学推理任务上取得实质性进步，同时保持了较高的数据效率。分析显示，数据合成与RLVR协同作用，共同促进了LLMs的自适应推理能力。后续分析进一步揭示了关键因素的影响机制及在指导LLMs中的应用价值。本项目已开源，访问地址为：https://github.com/LaiZhejian/AdaR。

English

Mathematical reasoning is a primary indicator of large language models (LLMs) intelligence. However, existing LLMs exhibit failures of robustness and generalization. This paper attributes these deficiencies to spurious reasoning, i.e., producing answers from superficial features. To address this challenge, we propose the AdaR framework to enable adaptive reasoning, wherein models rely on problem-solving logic to produce answers. AdaR synthesizes logically equivalent queries by varying variable values, and trains models with RLVR on these data to penalize spurious logic while encouraging adaptive logic. To improve data quality, we extract the problem-solving logic from the original query and generate the corresponding answer by code execution, then apply a sanity check. Experimental results demonstrate that AdaR improves robustness and generalization, achieving substantial improvement in mathematical reasoning while maintaining high data efficiency. Analysis indicates that data synthesis and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs. Subsequent analyses derive key design insights into the effect of critical factors and the applicability to instruct LLMs. Our project is available at https://github.com/LaiZhejian/AdaR

实现数学推理的自适应性

Making Mathematical Reasoning Adaptive

摘要

Support