ChatPaper.aiChatPaper

ChatGPT引导的编辑教练,用于抽象摘要定制化

ChatGPT-steered Editing Instructor for Customization of Abstractive Summarization

May 4, 2023
作者: Wen Xiao, Yujia Xie, Giuseppe Carenini, Pengcheng He
cs.AI

摘要

尽管大型语言模型(如ChatGPT)生成质量令人印象深刻,但将其输出定制化以满足特定用户需求仍然是一个挑战。在本文中,我们提出了一个三代理生成流程,包括一个生成器、一个指导者和一个编辑器,以增强生成输出的定制化。生成器产生初始输出,用户特定的指导者生成编辑指令,编辑器生成符合用户偏好的修订输出。仅推理的大型语言模型(ChatGPT)既充当生成器又充当编辑器,而较小的模型则充当用户特定的指导者,引导生成过程以满足用户需求。指导者使用编辑驱动的强化学习进行训练,利用来自大规模编辑器模型的反馈来优化指令生成。在两个抽象总结数据集上的实验结果表明,我们的方法在生成更符合用户期望的输出方面是有效的。
English
Tailoring outputs of large language models, such as ChatGPT, to specific user needs remains a challenge despite their impressive generation quality. In this paper, we propose a tri-agent generation pipeline consisting of a generator, an instructor, and an editor to enhance the customization of generated outputs. The generator produces an initial output, the user-specific instructor generates editing instructions, and the editor generates a revised output aligned with user preferences. The inference-only large language model (ChatGPT) serves as both the generator and the editor, while a smaller model acts as the user-specific instructor to guide the generation process toward user needs. The instructor is trained using editor-steered reinforcement learning, leveraging feedback from the large-scale editor model to optimize instruction generation. Experimental results on two abstractive summarization datasets demonstrate the effectiveness of our approach in generating outputs that better fulfill user expectations.
PDF31December 15, 2024