教会大语言模型个性化——一种受写作教育启发的路径

摘要

个性化文本生成是近年来备受关注的新兴研究领域。该方向的大多数研究通过设计定制化特征或模型聚焦于特定领域。本文提出了一种基于大语言模型（LLM）的通用个性化文本生成方法。受写作教育实践的启发，我们开发了一个多阶段、多任务的框架来训练大语言模型实现个性化生成。在写作教学中，基于素材的写作任务常被分解为多个步骤，包括信息查找、评估、总结、整合与融合。类似地，我们的个性化文本生成方法包含检索、排序、摘要、整合和生成多个阶段。此外，我们引入了多任务学习机制以进一步提升模型生成能力，其灵感来源于教育领域的观察——学生的阅读能力与写作水平往往具有相关性。我们在三个涵盖不同代表性领域的公开数据集上评估了该方法，实验结果表明相较于多种基线模型，本方法取得了显著提升。

English

Personalized text generation is an emerging research area that has attracted much attention in recent years. Most studies in this direction focus on a particular domain by designing bespoke features or models. In this work, we propose a general approach for personalized text generation using large language models (LLMs). Inspired by the practice of writing education, we develop a multistage and multitask framework to teach LLMs for personalized generation. In writing instruction, the task of writing from sources is often decomposed into multiple steps that involve finding, evaluating, summarizing, synthesizing, and integrating information. Analogously, our approach to personalized text generation consists of multiple stages: retrieval, ranking, summarization, synthesis, and generation. In addition, we introduce a multitask setting that helps the model improve its generation ability further, which is inspired by the observation in education that a student's reading proficiency and writing ability are often correlated. We evaluate our approach on three public datasets, each of which covers a different and representative domain. Our results show significant improvements over a variety of baselines.

教会大语言模型个性化——一种受写作教育启发的路径

Teach LLMs to Personalize -- An Approach inspired by Writing Education

摘要

Support