大语言模型推理轨迹中的认知片段实现可解释的人类项目难度预测

摘要

预测人类题目难度是教育评估的核心，可靠的难度估计能支撑公平性和有效试题构建。现有方法常依赖昂贵的人工校准或基于题目文本的表征，对导致题目难度的认知过程提供的证据有限。我们认为难度不应仅被视为题目文本的属性，更应作为题目引发解题负担的可观测结果。大型推理模型（LRMs）通过推理轨迹提供可扩展的过程证据，但此类证据需结构化以支持可解释建模。为此，我们提出Epi2Diff（从情节到难度）框架，该框架将LRM推理轨迹映射为基于认知的情结序列。这些情节将轨迹片段分组为功能性解题状态，从而通过推理规模、努力分配和状态转换对难度进行建模。Epi2Diff提取紧凑的情结动态特征，并将其与语义题目表征结合用于人类难度预测。在四个真实人类难度数据集上的实验表明，Epi2Diff持续优于强基线方法，包括微调的小型语言模型、大语言模型上下文学习和有监督大语言模型适配。在基于SAT的分类基准上，Epi2Diff相较于有监督大语言模型微调基线实现了8.1%的平均相对提升。进一步分析显示，较难题目会引发更费力、更迭代且以执行过程为中心的情结动态，而不仅仅是更长的回答。这些结果表明，LRM推理轨迹中的认知情结能为人类题目难度提供具预测性和可解释性的过程表征，为推理模型在教育测量中的应用提供了新视角。

English

Predicting human item difficulty is central to educational assessment, where reliable estimates support fairness and effective test construction. Existing methods often depend on costly human calibration or item-level textual representations, providing limited evidence about the cognitive processes that make items difficult. We argue that difficulty should be viewed not only as a property of item text, but also as an observable consequence of the problem-solving burden an item induces. Large Reasoning Models (LRMs) offer scalable process evidence through reasoning traces, but such evidence must be structured to support interpretable modeling. To this end, we introduce Epi2Diff (Episode to Difficulty), a framework that maps LRM reasoning traces into cognitively grounded episode sequences. These episodes group trace segments into functional problem-solving states, enabling difficulty to be modeled through reasoning scale, effort allocation, and state transitions. Epi2Diff extracts compact episode-dynamic features and combines them with semantic item representations for human difficulty prediction. Experiments on four real-world human difficulty datasets show that Epi2Diff consistently outperforms strong baselines, including fine-tuned small language models, LLM in-context learning, and supervised LLM adaptation. On SAT-derived classification benchmarks, Epi2Diff achieves an 8.1% average relative gain over supervised LLM fine-tuning baselines. Further analyses show that harder items induce more effortful, iterative, and implementation-centered episode dynamics, rather than merely longer responses. These results demonstrate that cognitive episodes in LRM reasoning traces provide a predictive and interpretable process representation for human item difficulty, offering a new lens for educational measurement with reasoning models.