ChatGPT引导的编辑教练，用于抽象摘要定制化

摘要

尽管大型语言模型（如ChatGPT）生成质量令人印象深刻，但将其输出定制化以满足特定用户需求仍然是一个挑战。在本文中，我们提出了一个三代理生成流程，包括一个生成器、一个指导者和一个编辑器，以增强生成输出的定制化。生成器产生初始输出，用户特定的指导者生成编辑指令，编辑器生成符合用户偏好的修订输出。仅推理的大型语言模型（ChatGPT）既充当生成器又充当编辑器，而较小的模型则充当用户特定的指导者，引导生成过程以满足用户需求。指导者使用编辑驱动的强化学习进行训练，利用来自大规模编辑器模型的反馈来优化指令生成。在两个抽象总结数据集上的实验结果表明，我们的方法在生成更符合用户期望的输出方面是有效的。

English

Tailoring outputs of large language models, such as ChatGPT, to specific user needs remains a challenge despite their impressive generation quality. In this paper, we propose a tri-agent generation pipeline consisting of a generator, an instructor, and an editor to enhance the customization of generated outputs. The generator produces an initial output, the user-specific instructor generates editing instructions, and the editor generates a revised output aligned with user preferences. The inference-only large language model (ChatGPT) serves as both the generator and the editor, while a smaller model acts as the user-specific instructor to guide the generation process toward user needs. The instructor is trained using editor-steered reinforcement learning, leveraging feedback from the large-scale editor model to optimize instruction generation. Experimental results on two abstractive summarization datasets demonstrate the effectiveness of our approach in generating outputs that better fulfill user expectations.