MathFusion：透過指令融合提升大型語言模型的數學問題解決能力

MathFusion: Enhancing Mathematic Problem-solving of LLM through Instruction Fusion

March 20, 2025

作者: Qizhi Pei, Lijun Wu, Zhuoshi Pan, Yu Li, Honglin Lin, Chenlin Ming, Xin Gao, Conghui He, Rui Yan

cs.AI

摘要

大型语言模型（LLMs）在数学推理方面展现了显著的进步。尽管数据增强有望提升数学问题解决能力，但当前方法主要局限于实例层面的修改——如重述或生成句法变体——这些方法未能捕捉并利用数学知识中固有的关系结构。受人类学习过程的启发，其中数学能力通过系统性地接触相互关联的概念而发展，我们引入了MathFusion，一个通过跨问题指令合成来增强数学推理的新框架。MathFusion通过三种融合策略实现这一目标：（1）顺序融合，将相关问题串联以建模解决方案的依赖关系；（2）并行融合，结合类似问题以强化概念理解；（3）条件融合，创建上下文感知的选择性问题以增强推理的灵活性。通过应用这些策略，我们生成了新数据集MathFusionQA，并在此基础上微调了模型（DeepSeekMath-7B、Mistral-7B、Llama3-8B）。实验结果表明，MathFusion在保持高数据效率的同时，显著提升了数学推理能力，在多样化的基准测试中准确率提高了18.0个百分点，而仅需45K额外的合成指令，相较于传统的单一指令方法实现了显著改进。我们的数据集、模型和代码已公开于https://github.com/QizhiPei/mathfusion。

English

Large Language Models (LLMs) have shown impressive progress in mathematical reasoning. While data augmentation is promising to enhance mathematical problem-solving ability, current approaches are predominantly limited to instance-level modifications-such as rephrasing or generating syntactic variations-which fail to capture and leverage the intrinsic relational structures inherent in mathematical knowledge. Inspired by human learning processes, where mathematical proficiency develops through systematic exposure to interconnected concepts, we introduce MathFusion, a novel framework that enhances mathematical reasoning through cross-problem instruction synthesis. MathFusion implements this through three fusion strategies: (1) sequential fusion, which chains related problems to model solution dependencies; (2) parallel fusion, which combines analogous problems to reinforce conceptual understanding; and (3) conditional fusion, which creates context-aware selective problems to enhance reasoning flexibility. By applying these strategies, we generate a new dataset, MathFusionQA, followed by fine-tuning models (DeepSeekMath-7B, Mistral-7B, Llama3-8B) on it. Experimental results demonstrate that MathFusion achieves substantial improvements in mathematical reasoning while maintaining high data efficiency, boosting performance by 18.0 points in accuracy across diverse benchmarks while requiring only 45K additional synthetic instructions, representing a substantial improvement over traditional single-instruction approaches. Our datasets, models, and code are publicly available at https://github.com/QizhiPei/mathfusion.