ChatGPT引导的编辑教练,用于抽象摘要定制化
ChatGPT-steered Editing Instructor for Customization of Abstractive Summarization
May 4, 2023
作者: Wen Xiao, Yujia Xie, Giuseppe Carenini, Pengcheng He
cs.AI
摘要
尽管大型语言模型(如ChatGPT)生成质量令人印象深刻,但将其输出定制化以满足特定用户需求仍然是一个挑战。在本文中,我们提出了一个三代理生成流程,包括一个生成器、一个指导者和一个编辑器,以增强生成输出的定制化。生成器产生初始输出,用户特定的指导者生成编辑指令,编辑器生成符合用户偏好的修订输出。仅推理的大型语言模型(ChatGPT)既充当生成器又充当编辑器,而较小的模型则充当用户特定的指导者,引导生成过程以满足用户需求。指导者使用编辑驱动的强化学习进行训练,利用来自大规模编辑器模型的反馈来优化指令生成。在两个抽象总结数据集上的实验结果表明,我们的方法在生成更符合用户期望的输出方面是有效的。
English
Tailoring outputs of large language models, such as ChatGPT, to specific user
needs remains a challenge despite their impressive generation quality. In this
paper, we propose a tri-agent generation pipeline consisting of a generator, an
instructor, and an editor to enhance the customization of generated outputs.
The generator produces an initial output, the user-specific instructor
generates editing instructions, and the editor generates a revised output
aligned with user preferences. The inference-only large language model
(ChatGPT) serves as both the generator and the editor, while a smaller model
acts as the user-specific instructor to guide the generation process toward
user needs. The instructor is trained using editor-steered reinforcement
learning, leveraging feedback from the large-scale editor model to optimize
instruction generation. Experimental results on two abstractive summarization
datasets demonstrate the effectiveness of our approach in generating outputs
that better fulfill user expectations.