大型語言模型的可控文本生成：一項調查

Controllable Text Generation for Large Language Models: A Survey

August 22, 2024

作者: Xun Liang, Hanyu Wang, Yezhaohui Wang, Shichao Song, Jiawei Yang, Simin Niu, Jie Hu, Dan Liu, Shunyu Yao, Feiyu Xiong, Zhiyu Li

cs.AI

摘要

在自然語言處理（NLP）中，大型語言模型（LLMs）展現出高質量的文本生成能力。然而，在實際應用中，LLMs必須滿足日益複雜的需求。除了避免誤導或不當內容外，LLMs還應該滿足特定用戶需求，例如模仿特定的寫作風格或生成具有詩意豐富性的文本。這些多樣化的需求推動了可控文本生成（CTG）技術的發展，確保輸出符合預定的控制條件，如安全性、情感、主題一致性和語言風格，同時保持高水準的幫助性、流暢性和多樣性。 本文系統地回顧了LLMs的CTG的最新進展，提供了其核心概念的全面定義，並澄清了控制條件和文本質量的要求。我們將CTG任務分為兩種主要類型：內容控制和屬性控制。討論了關鍵方法，包括模型重新訓練、微調、強化學習、提示工程、潛在空間操作和解碼時間干預。我們分析了每種方法的特點、優勢和局限性，提供了實珵洞察，以實現生成控制。此外，我們還回顧了CTG的評估方法，總結了其在各個領域的應用，並解決了當前研究中的關鍵挑戰，包括流暢性和實用性的降低。我們還提出了一些建議，例如在未來研究中更加重視實際應用。本文旨在為該領域的研究人員和開發人員提供有價值的指導。我們的參考文獻和中文版本均在https://github.com/IAAR-Shanghai/CTGSurvey上開源。

English

In Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated high text generation quality. However, in real-world applications, LLMs must meet increasingly complex requirements. Beyond avoiding misleading or inappropriate content, LLMs are also expected to cater to specific user needs, such as imitating particular writing styles or generating text with poetic richness. These varied demands have driven the development of Controllable Text Generation (CTG) techniques, which ensure that outputs adhere to predefined control conditions--such as safety, sentiment, thematic consistency, and linguistic style--while maintaining high standards of helpfulness, fluency, and diversity. This paper systematically reviews the latest advancements in CTG for LLMs, offering a comprehensive definition of its core concepts and clarifying the requirements for control conditions and text quality. We categorize CTG tasks into two primary types: content control and attribute control. The key methods are discussed, including model retraining, fine-tuning, reinforcement learning, prompt engineering, latent space manipulation, and decoding-time intervention. We analyze each method's characteristics, advantages, and limitations, providing nuanced insights for achieving generation control. Additionally, we review CTG evaluation methods, summarize its applications across domains, and address key challenges in current research, including reduced fluency and practicality. We also propose several appeals, such as placing greater emphasis on real-world applications in future research. This paper aims to offer valuable guidance to researchers and developers in the field. Our reference list and Chinese version are open-sourced at https://github.com/IAAR-Shanghai/CTGSurvey.