规划者：通过潜在语言扩散生成多样化段落模型

摘要

基于自回归模型的文本有时会生成重复且质量低下的输出，因为在生成过程中错误会累积。这个问题通常被归因于曝光偏差 - 模型在训练时和在推断过程中的使用方式之间的差异。去噪扩散模型提供了一种替代方法，其中模型可以重新访问和修订其输出。然而，它们可能在计算上昂贵，以往在文本方面的努力导致的模型产生的流畅性较差，特别是对于较长的文本和段落。在本文中，我们提出了PLANNER，这是一个将潜在语义扩散与自回归生成相结合的模型，以在段落上全局控制的同时生成流畅的文本。该模型通过将自回归的“解码”模块与使用潜在扩散以粗到细的方式生成语义段落嵌入的“规划”模块相结合来实现这一目标。所提出的方法在各种条件生成任务上进行了评估，语义生成、文本补全和摘要的结果显示了其在高效生成高质量长文本方面的有效性。

English

Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation. This issue is often attributed to exposure bias - the difference between how a model is trained, and how it is used during inference. Denoising diffusion models provide an alternative approach in which a model can revisit and revise its output. However, they can be computationally expensive and prior efforts on text have led to models that produce less fluent output compared to autoregressive models, especially for longer text and paragraphs. In this paper, we propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation, to generate fluent text while exercising global control over paragraphs. The model achieves this by combining an autoregressive "decoding" module with a "planning" module that uses latent diffusion to generate semantic paragraph embeddings in a coarse-to-fine manner. The proposed method is evaluated on various conditional generation tasks, and results on semantic generation, text completion and summarization show its effectiveness in generating high-quality long-form text in an efficient manner.

规划者：通过潜在语言扩散生成多样化段落模型

PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model

摘要

Support