我们在测量策略还是措辞？——LLM数学推理中表层多样性与方法层面多样性之间的差距

摘要

LLM数学推理中的多样性对于探索至关重要，但常见的多样性指标大多仅捕捉表面差异，而非解题策略的根本不同。为填补这一空白，我们引入了方法层面多样性：同一问题正确解法在策略上的差异。借助经人工校准的LLM评判框架，我们证明现有多样性度量无法可靠表征方法层面多样性，且这一错位会延续至多样性感知的RLVR中——目标指标得以保留，而方法层面多样性却持续下降。通过探究方法层面多样性何时产生助力以及能否直接诱导，发现具备方法多样性的候选集能提升测试时扩展性能。然而，在训练过程中优化LLM评判的多样性奖励，会导致策略倾向于利用评判者的特定偏好，而非拓展解题方法，这使得方法层面多样性的直接优化仍为开放性问题。综上，本研究提出了方法层面多样性的概念，揭示了表面信号与方法层面信号之间的系统性背离，为促使LLM以真正多样化、类人方式推理迈出一步。

English

Diversity in LLM mathematical reasoning is critical for exploration, but common diversity metrics mostly capture surface-level variation rather than differences in how a problem is solved. We address this gap by introducing approach-level diversity: variation in strategies across correct solutions to the same problem. Using a human-calibrated LLM judge framework, we show that prior diversity measures are unreliable proxies for approach-level diversity, and this mismatch carries over to diversity-aware RLVR, where target metrics are preserved while approach-level diversity declines. Investigating when approach-level diversity helps and whether it can be directly induced, we find that approach-diverse candidate sets improve test-time scaling. However, optimizing an LLM judge diversity reward during training causes the policy to exploit judge-specific preferences rather than broaden its approaches, leaving direct optimization of approach-level diversity as an open problem. Together, our work introduces the notion of approach-level diversity and uncovers a systematic divergence between surface- and approach-level signals, marking a step toward LLMs that reason in genuinely diverse, human-like ways.