DiagEvo：基于分层错误记忆的诊断引导自进化

摘要

自博弈是语言模型自我进化的有效范式，但在缺乏引导的情况下，求解器的性能可能随轮次推进而停滞甚至下降。无引导方法通过难度、可学习性或多样性等信号驱动问题生成，这些信号使问题保持挑战性和多样性，但并未指明后续轮次应针对哪些未解决的内在推理薄弱点。有引导方法从外部任务资源（如人工示例、文档语料库或指定的难度目标）获取方向，因此依赖于自博弈循环之外提供的任务信息。我们证明所需的引导可以从求解器自身的失败历史中推导出来。我们提出了DiagEvo，其诊断器从失败历史中提取反复出现的错误原因，并将其存储于层级化的错误原因记忆中。该记忆将相关原因归组于技能节点之下，并根据针对性问题上的自一致性将每个节点标记为“活跃”或“已掌握”。挑战者利用这些状态及复发次数，在原因定向生成与自由探索之间取得平衡。双重置信度过滤仅在最常见求解器答案具有明显票数领先优势时，保留中等难度问题。DiagEvo从自博弈过程中产生的信息推导其课程，无需任何外部任务资源。在默认的4B诊断器配置下，对于三种求解器——Qwen3-4B、Qwen3-8B和OctoThinker-8B——DiagEvo在全部九个基准上的平均准确率均优于所有基线。在Qwen3-8B上，DiagEvo在五个数学推理基准上达到72.3%的平均准确率，比R-Zero高出4.5个百分点；在全部九个基准上的平均准确率为57.4%，比DARC高出1.1个百分点。消融实验表明，层级化错误原因记忆与双重置信度过滤均对此增益有所贡献。

English

Self-play is an effective paradigm for language-model self-evolution, but without guidance, solver performance can plateau or decline across rounds. Unguided methods steer question generation with signals such as difficulty, learnability, or diversity. These signals keep questions challenging and varied but do not specify which unresolved reasoning weaknesses later rounds should target. Guided methods obtain direction from external task resources, including human examples, document corpora, or specified difficulty targets, and therefore rely on task information supplied outside the self-play loop. We show that the needed direction can instead be derived from the solver's own failure history. We introduce DiagEvo, whose diagnostician extracts recurring error causes from this history and stores them in a hierarchical error-cause memory. The memory groups related causes under skill nodes and tracks each as Active or Mastered according to self-consistency on targeted questions. The challenger uses these states and recurrence counts to balance cause-targeted generation with free exploration. Double-confidence filtering retains intermediate-difficulty questions only when the most common solver answer has a clear vote lead. DiagEvo derives its curriculum from information produced during self-play, without external task resources. With the default 4B diagnostician, DiagEvo outperforms every baseline in mean accuracy across all nine benchmarks for each of the three solvers: Qwen3-4B, Qwen3-8B, and OctoThinker-8B. On Qwen3-8B, it reaches 72.3% mean accuracy across five mathematical reasoning benchmarks, 4.5 percentage points above R-Zero. Its mean accuracy across all nine benchmarks is 57.4%, 1.1 percentage points above DARC. Ablations show that the hierarchical error-cause memory and double-confidence filtering both contribute to these gains.