EquivPruner:通過動作剪枝提升基於LLM搜索的效率與質量
EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning
May 22, 2025
作者: Jiawei Liu, Qisi Chen, Jianshu Zhang, Quan Liu, Defu Lian
cs.AI
摘要
大型语言模型(LLMs)在通过搜索算法进行复杂推理方面表现出色,然而当前的策略常因对语义等价步骤的冗余探索而消耗大量标记。现有的语义相似性方法在特定领域(如数学推理)中难以准确识别此类等价性。为此,我们提出了EquivPruner,一种简单而有效的方法,旨在LLM推理搜索过程中识别并剪枝语义等价的动作。同时,我们引入了MathEquiv,这是首个为数学陈述等价性创建的数据集,它支持训练一个轻量级的等价性检测器。跨多种模型和任务的广泛实验表明,EquivPruner显著减少了标记消耗,提升了搜索效率,并时常增强了推理准确性。例如,在将EquivPruner应用于Qwen2.5-Math-7B-Instruct模型处理GSM8K数据集时,标记消耗减少了48.1%,同时准确性也有所提高。我们的代码已发布于https://github.com/Lolo1222/EquivPruner。
English
Large Language Models (LLMs) excel at complex reasoning through search
algorithms, yet current strategies often suffer from massive token consumption
due to redundant exploration of semantically equivalent steps. Existing
semantic similarity methods struggle to accurately identify such equivalence in
domain-specific contexts like mathematical reasoning. To address this, we
propose EquivPruner, a simple yet effective approach that identifies and prunes
semantically equivalent actions during LLM reasoning search. We also introduce
MathEquiv, the first dataset we created for mathematical statement equivalence,
which enables the training of a lightweight equivalence detector. Extensive
experiments across various models and tasks demonstrate that EquivPruner
significantly reduces token consumption, improving searching efficiency and
often bolstering reasoning accuracy. For instance, when applied to
Qwen2.5-Math-7B-Instruct on GSM8K, EquivPruner reduced token consumption by
48.1\% while also improving accuracy. Our code is available at
https://github.com/Lolo1222/EquivPruner.Summary
AI-Generated Summary