教導大型語言模型實現個性化——一種受寫作教育啟發的方法

摘要

個性化文本生成是近年來備受關注的新興研究領域。該方向的多數研究聚焦於特定領域，通過設計專用特徵或模型實現個性化生成。本文提出一種基於大型語言模型的通用個性化文本生成方法。受寫作教學實踐啟發，我們開發了多階段多任務框架來訓練大型語言模型進行個性化生成。在寫作教學中，基於素材的寫作任務常被分解為多個步驟：查找、評估、總結、綜合及整合信息。類比地，我們的個性化文本生成方法包含檢索、排序、摘要、合成與生成五個階段。此外，我們引入多任務學習機制以進一步提升模型生成能力，這源於教育領域中觀察到的閱讀能力與寫作水平常具相關性的現象。我們在三個涵蓋不同代表性領域的公開數據集上進行評估，結果表明該方法相較多種基準模型均有顯著提升。

English

Personalized text generation is an emerging research area that has attracted much attention in recent years. Most studies in this direction focus on a particular domain by designing bespoke features or models. In this work, we propose a general approach for personalized text generation using large language models (LLMs). Inspired by the practice of writing education, we develop a multistage and multitask framework to teach LLMs for personalized generation. In writing instruction, the task of writing from sources is often decomposed into multiple steps that involve finding, evaluating, summarizing, synthesizing, and integrating information. Analogously, our approach to personalized text generation consists of multiple stages: retrieval, ranking, summarization, synthesis, and generation. In addition, we introduce a multitask setting that helps the model improve its generation ability further, which is inspired by the observation in education that a student's reading proficiency and writing ability are often correlated. We evaluate our approach on three public datasets, each of which covers a different and representative domain. Our results show significant improvements over a variety of baselines.

教導大型語言模型實現個性化——一種受寫作教育啟發的方法

Teach LLMs to Personalize -- An Approach inspired by Writing Education

摘要

Support