策略：通过轨迹调制游戏自对弈学习可迁移推理能力

摘要

游戏为开发语言模型的通用推理能力提供了引人入胜的范式，因其天然需要战略规划、概率推断和自适应决策。然而，现有自我博弈方法仅依赖最终游戏结果，无法区分可迁移的推理模式与游戏特定启发式策略。我们提出的STRATAGEM方案旨在解决推理迁移的两大根本障碍：领域特定性（即习得模式始终受限于游戏语义）和语境固化（即静态游戏环境无法培养渐进式推理）。该方案通过推理可迁移性系数选择性强化展现抽象、领域无关推理的轨迹，同时借助推理进化奖励激励自适应推理能力的发展。在数学推理、通用推理和代码生成的基准测试中，实验结果表明该方法带来显著提升，尤其在需要多步推理的竞赛级数学问题上表现突出。消融研究与人工评估证实，两个组件共同促进了可迁移推理能力的形成。

English

Games offer a compelling paradigm for developing general reasoning capabilities in language models, as they naturally demand strategic planning, probabilistic inference, and adaptive decision-making. However, existing self-play approaches rely solely on terminal game outcomes, providing no mechanism to distinguish transferable reasoning patterns from game-specific heuristics. We present STRATAGEM, which addresses two fundamental barriers to reasoning transfer: domain specificity, where learned patterns remain anchored in game semantics, and contextual stasis, where static game contexts fail to cultivate progressive reasoning. STRATAGEM selectively reinforces trajectories exhibiting abstract, domain-agnostic reasoning through a Reasoning Transferability Coefficient, while incentivizing adaptive reasoning development via a Reasoning Evolution Reward. Experiments across mathematical reasoning, general reasoning, and code generation benchmarks demonstrate substantial improvements, with particularly strong gains on competition-level mathematics where multi-step reasoning is critical. Ablation studies and human evaluation confirm that both components contribute to transferable reasoning.

策略：通过轨迹调制游戏自对弈学习可迁移推理能力

Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play

摘要

Support