使數學推理具備適應性

摘要

數學推理是大型語言模型（LLMs）智能的主要指標。然而，現有的LLMs在魯棒性和泛化能力方面存在不足。本文將這些缺陷歸因於虛假推理，即從表面特徵生成答案。為應對這一挑戰，我們提出了AdaR框架，以實現自適應推理，使模型依賴於問題解決邏輯來產生答案。AdaR通過改變變量值來合成邏輯等價的查詢，並利用RLVR在這些數據上訓練模型，以懲罰虛假邏輯，同時鼓勵自適應邏輯。為提高數據質量，我們從原始查詢中提取問題解決邏輯，並通過代碼執行生成相應答案，然後進行合理性檢查。實驗結果表明，AdaR提升了魯棒性和泛化能力，在數學推理方面取得了顯著進步，同時保持了高數據效率。分析表明，數據合成和RLVR協同作用，促進了LLMs中的自適應推理。後續分析得出了關鍵設計洞察，涉及關鍵因素的影響以及對LLMs指令的適用性。我們的項目可在https://github.com/LaiZhejian/AdaR獲取。

English

Mathematical reasoning is a primary indicator of large language models (LLMs) intelligence. However, existing LLMs exhibit failures of robustness and generalization. This paper attributes these deficiencies to spurious reasoning, i.e., producing answers from superficial features. To address this challenge, we propose the AdaR framework to enable adaptive reasoning, wherein models rely on problem-solving logic to produce answers. AdaR synthesizes logically equivalent queries by varying variable values, and trains models with RLVR on these data to penalize spurious logic while encouraging adaptive logic. To improve data quality, we extract the problem-solving logic from the original query and generate the corresponding answer by code execution, then apply a sanity check. Experimental results demonstrate that AdaR improves robustness and generalization, achieving substantial improvement in mathematical reasoning while maintaining high data efficiency. Analysis indicates that data synthesis and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs. Subsequent analyses derive key design insights into the effect of critical factors and the applicability to instruct LLMs. Our project is available at https://github.com/LaiZhejian/AdaR

使數學推理具備適應性

Making Mathematical Reasoning Adaptive

摘要

Support