导师协助：一种用于扩展实时专业知识的人工智能方法

Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise

October 3, 2024

作者: Rose E. Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, Dora Demszky

cs.AI

摘要

生成式人工智能，特别是语言模型（LMs），有潜力改变具有社会影响的现实领域，特别是在专家资源有限的情况下。例如，在教育领域，用专家指导培训新手教育工作者对提高教育质量至关重要，但成本高昂，从而在规模上改善教育质量存在重大障碍。这一挑战对来自弱势社区的学生造成不成比例的伤害，而这些学生最有可能从高质量教育中获益。我们介绍了Tutor CoPilot，这是一种新颖的人工智能方法，利用专家思维模型为导师提供类似专家的指导。这项研究是在实时辅导中进行的第一项人工智能系统的随机对照试验，涉及来自历史上被忽视社区的900名导师和1,800名K-12学生。根据预先注册的分析计划，我们发现与使用Tutor CoPilot的导师合作的学生更有可能掌握主题（p<0.01）的概率高出4个百分点。值得注意的是，评级较低的导师的学生获益最大，掌握程度提高了9个百分点。我们发现，Tutor CoPilot每年每位导师的成本仅为20美元。我们使用分类器分析了55万条以上的消息，以识别教学策略，并发现使用Tutor CoPilot的导师更有可能使用高质量策略促进学生理解（例如，提出引导性问题），并且更不太可能直接给出答案。导师访谈突出了Tutor CoPilot的指导如何帮助导师应对学生需求，尽管他们指出了Tutor CoPilot存在的问题，例如生成的建议不适合年级水平。总的来说，我们对Tutor CoPilot的研究展示了人工智能系统如何在现实领域扩展专业知识，弥合技能差距，并创造一个未来，让所有学生都能获得高质量教育。

English

Generative AI, particularly Language Models (LMs), has the potential to transform real-world domains with societal impact, particularly where access to experts is limited. For example, in education, training novice educators with expert guidance is important for effectiveness but expensive, creating significant barriers to improving education quality at scale. This challenge disproportionately harms students from under-served communities, who stand to gain the most from high-quality education. We introduce Tutor CoPilot, a novel Human-AI approach that leverages a model of expert thinking to provide expert-like guidance to tutors as they tutor. This study is the first randomized controlled trial of a Human-AI system in live tutoring, involving 900 tutors and 1,800 K-12 students from historically under-served communities. Following a preregistered analysis plan, we find that students working with tutors that have access to Tutor CoPilot are 4 percentage points (p.p.) more likely to master topics (p<0.01). Notably, students of lower-rated tutors experienced the greatest benefit, improving mastery by 9 p.p. We find that Tutor CoPilot costs only $20 per-tutor annually. We analyze 550,000+ messages using classifiers to identify pedagogical strategies, and find that tutors with access to Tutor CoPilot are more likely to use high-quality strategies to foster student understanding (e.g., asking guiding questions) and less likely to give away the answer to the student. Tutor interviews highlight how Tutor CoPilot's guidance helps tutors to respond to student needs, though they flag issues in Tutor CoPilot, such as generating suggestions that are not grade-level appropriate. Altogether, our study of Tutor CoPilot demonstrates how Human-AI systems can scale expertise in real-world domains, bridge gaps in skills and create a future where high-quality education is accessible to all students.