我能为您点单吗？基于蒙特卡洛树搜索的扩散语言模型槽位填充排序方法

摘要

尽管掩码扩散模型（MDMs）中的规划-填充解码方法在数学与代码推理任务中展现出潜力，但其性能对槽位填充顺序高度敏感，常导致显著的输出差异。我们提出McDiffuSE框架，将槽位选择建模为决策过程，并通过蒙特卡洛树搜索（MCTS）优化填充顺序。该框架通过前瞻模拟评估部分生成结果，系统性地探索生成顺序的组合空间。实验表明，该方法相比自回归基线平均提升3.2%，较基线规划-填充方法提升8.0%，在MBPP和MATH500数据集上分别取得19.5%和4.9%的显著增益。分析表明，虽然McDiffuSE主要遵循顺序生成模式，但融入非顺序生成对最大化性能至关重要。我们发现，需通过增大探索常数（而非增加模拟次数）来克服模型置信度偏差，从而发现有效生成顺序。这些发现确立了基于MCTS的规划作为提升MDMs生成质量的有效途径。

English

While plan-and-infill decoding in Masked Diffusion Models (MDMs) shows promise for mathematical and code reasoning, performance remains highly sensitive to slot infilling order, often yielding substantial output variance. We introduce McDiffuSE, a framework that formulates slot selection as decision making and optimises infilling orders through Monte Carlo Tree Search (MCTS). McDiffuSE uses look-ahead simulations to evaluate partial completions before commitment, systematically exploring the combinatorial space of generation orders. Experiments show an average improvement of 3.2% over autoregressive baselines and 8.0% over baseline plan-and-infill, with notable gains of 19.5% on MBPP and 4.9% on MATH500. Our analysis reveals that while McDiffuSE predominantly follows sequential ordering, incorporating non-sequential generation is essential for maximising performance. We observe that larger exploration constants, rather than increased simulations, are necessary to overcome model confidence biases and discover effective orderings. These findings establish MCTS-based planning as an effective approach for enhancing generation quality in MDMs.

我能为您点单吗？基于蒙特卡洛树搜索的扩散语言模型槽位填充排序方法

Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models

摘要

Support