IntrEx：教育对话中参与度建模的数据集

摘要

在第二语言习得过程中，参与度和动机至关重要，然而在教育对话中保持学习者的兴趣仍是一大挑战。尽管先前的研究已探讨了教育文本的有趣性因素，但对于驱动对话参与度的语言特征仍知之甚少。为填补这一空白，我们引入了IntrEx，这是首个针对师生互动中有趣度及预期有趣度进行标注的大规模数据集。基于教师-学生聊天室语料库（TSCC），IntrEx通过引入序列级标注扩展了先前的工作，使得研究能够超越单轮对话，捕捉兴趣在长对话中的演变过程。我们采用了一套严格的标注流程，邀请了超过100名第二语言学习者参与，并借鉴了基于人类反馈的强化学习（RLHF）中的比较评分方法，以提高标注一致性。我们探究了大语言模型（LLMs）能否预测人类对有趣度的判断。研究发现，经过有趣度评分微调的LLMs（7B/8B参数）在预测效果上超越了如GPT-4o等更大的专有模型，展示了专门数据集在教育场景中建模参与度的潜力。最后，我们分析了具体性、可理解性（可读性）及吸收度等语言与认知因素如何影响教育对话中的参与度。

English

Engagement and motivation are crucial for second-language acquisition, yet maintaining learner interest in educational conversations remains a challenge. While prior research has explored what makes educational texts interesting, still little is known about the linguistic features that drive engagement in conversations. To address this gap, we introduce IntrEx, the first large dataset annotated for interestingness and expected interestingness in teacher-student interactions. Built upon the Teacher-Student Chatroom Corpus (TSCC), IntrEx extends prior work by incorporating sequence-level annotations, allowing for the study of engagement beyond isolated turns to capture how interest evolves over extended dialogues. We employ a rigorous annotation process with over 100 second-language learners, using a comparison-based rating approach inspired by reinforcement learning from human feedback (RLHF) to improve agreement. We investigate whether large language models (LLMs) can predict human interestingness judgments. We find that LLMs (7B/8B parameters) fine-tuned on interestingness ratings outperform larger proprietary models like GPT-4o, demonstrating the potential for specialised datasets to model engagement in educational settings. Finally, we analyze how linguistic and cognitive factors, such as concreteness, comprehensibility (readability), and uptake, influence engagement in educational dialogues.

IntrEx：教育对话中参与度建模的数据集

IntrEx: A Dataset for Modeling Engagement in Educational Conversations

摘要

Support