LLM能通过教学来学习吗？初步研究

Can LLMs Learn by Teaching? A Preliminary Study

June 20, 2024

作者: Xuefei Ning, Zifu Wang, Shiyao Li, Zinan Lin, Peiran Yao, Tianyu Fu, Matthew B. Blaschko, Guohao Dai, Huazhong Yang, Yu Wang

cs.AI

摘要

教学以改进学生模型（例如知识蒸馏）是LLM中广泛研究的方法论。然而，对于人类而言，教学不仅改善了学生，也提升了教师自身。我们提出一个问题：LLM是否也能通过教学来学习（LbT）？如果是，我们可能可以解锁在不仅仅依赖人类生成数据或更强模型的情况下，持续推进模型的可能性。在本文中，我们对这一宏伟议程进行初步探索。我们展示了LbT思想可以融入现有LLM训练/提示流程中，并带来显著改进。具体而言，我们设计了三种方法，分别模仿人类LbT的三个层次：观察学生的反馈、从反馈中学习以及迭代学习，旨在提高答案准确性而无需训练，并通过微调提高模型固有能力。研究结果令人鼓舞。例如，类似于人类的LbT，我们发现：（1）LbT可以引发弱到强的泛化：强模型可以通过教授其他弱模型来改进自身；（2）学生的多样性可能有助于：教授多个学生可能比教授一个学生或教师本身更好。我们希望这一早期的前景能激发对LbT的未来研究，并更广泛地采用教育中的先进技术来改进LLM。代码可在https://github.com/imagination-research/lbt找到。

English

Teaching to improve student models (e.g., knowledge distillation) is an extensively studied methodology in LLMs. However, for humans, teaching not only improves students but also improves teachers. We ask: Can LLMs also learn by teaching (LbT)? If yes, we can potentially unlock the possibility of continuously advancing the models without solely relying on human-produced data or stronger models. In this paper, we provide a preliminary exploration of this ambitious agenda. We show that LbT ideas can be incorporated into existing LLM training/prompting pipelines and provide noticeable improvements. Specifically, we design three methods, each mimicking one of the three levels of LbT in humans: observing students' feedback, learning from the feedback, and learning iteratively, with the goals of improving answer accuracy without training and improving models' inherent capability with fine-tuning. The findings are encouraging. For example, similar to LbT in human, we see that: (1) LbT can induce weak-to-strong generalization: strong models can improve themselves by teaching other weak models; (2) Diversity in students might help: teaching multiple students could be better than teaching one student or the teacher itself. We hope that this early promise can inspire future research on LbT and more broadly adopting the advanced techniques in education to improve LLMs. The code is available at https://github.com/imagination-research/lbt.