IntrEx：教育對話中參與度建模的數據集

摘要

參與度和動機對於第二語言習得至關重要，然而在教育對話中保持學習者的興趣仍然是一大挑戰。儘管先前的研究已探討了教育文本的有趣性因素，但對於驅動對話參與度的語言特徵仍知之甚少。為填補這一空白，我們引入了IntrEx，這是首個針對師生互動中有趣性及預期有趣性進行註釋的大規模數據集。基於教師-學生聊天室語料庫（TSCC），IntrEx通過引入序列級別的註釋擴展了先前的工作，使得研究能夠超越單一輪次的參與度，捕捉興趣在長時間對話中的演變。我們採用嚴格的註釋流程，邀請超過100名第二語言學習者參與，並借鑒基於人類反饋的強化學習（RLHF）中的比較評分方法，以提高註釋一致性。我們探討了大語言模型（LLMs）是否能夠預測人類對有趣性的判斷。結果顯示，經過有趣性評分微調的LLMs（7B/8B參數）在性能上超越了如GPT-4o等更大的專有模型，這表明專門數據集在教育場景中建模參與度的潛力。最後，我們分析了具體性、可理解性（可讀性）及接納度等語言和認知因素如何影響教育對話中的參與度。

English

Engagement and motivation are crucial for second-language acquisition, yet maintaining learner interest in educational conversations remains a challenge. While prior research has explored what makes educational texts interesting, still little is known about the linguistic features that drive engagement in conversations. To address this gap, we introduce IntrEx, the first large dataset annotated for interestingness and expected interestingness in teacher-student interactions. Built upon the Teacher-Student Chatroom Corpus (TSCC), IntrEx extends prior work by incorporating sequence-level annotations, allowing for the study of engagement beyond isolated turns to capture how interest evolves over extended dialogues. We employ a rigorous annotation process with over 100 second-language learners, using a comparison-based rating approach inspired by reinforcement learning from human feedback (RLHF) to improve agreement. We investigate whether large language models (LLMs) can predict human interestingness judgments. We find that LLMs (7B/8B parameters) fine-tuned on interestingness ratings outperform larger proprietary models like GPT-4o, demonstrating the potential for specialised datasets to model engagement in educational settings. Finally, we analyze how linguistic and cognitive factors, such as concreteness, comprehensibility (readability), and uptake, influence engagement in educational dialogues.

IntrEx：教育對話中參與度建模的數據集

IntrEx: A Dataset for Modeling Engagement in Educational Conversations

摘要

Support