LayoutPrompter: 唤醒大型语言模型的设计能力

LayoutPrompter: Awaken the Design Ability of Large Language Models

November 11, 2023

作者: Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang James Yang, Jian-Guang Lou, Dongmei Zhang

cs.AI

摘要

条件图形布局生成自动将用户约束映射到高质量布局，如今已经引起了广泛关注。尽管最近的研究取得了令人期待的性能，但缺乏通用性和数据效率阻碍了它们的实际应用。在这项工作中，我们提出了LayoutPrompter，利用大型语言模型（LLMs）通过上下文学习来解决上述问题。LayoutPrompter由三个关键组件组成，即输入输出序列化、动态示例选择和布局排名。具体而言，输入输出序列化组件精心设计了每个布局生成任务的输入和输出格式。动态示例选择负责为给定输入选择最有帮助的提示示例。布局排名器用于从LLMs的多个输出中选择最高质量的布局。我们使用四个公共数据集对所有现有的布局生成任务进行实验。尽管我们的方法简单，实验结果表明LayoutPrompter在这些任务上可以与甚至胜过最先进的方法，而无需任何模型训练或微调。这证明了这种通用且无需训练的方法的有效性。此外，消融研究表明，在低数据情况下，LayoutPrompter明显优于基于训练的基线，进一步表明了LayoutPrompter的数据效率。我们的项目可在https://github.com/microsoft/LayoutGeneration/tree/main/LayoutPrompter找到。

English

Conditional graphic layout generation, which automatically maps user constraints to high-quality layouts, has attracted widespread attention today. Although recent works have achieved promising performance, the lack of versatility and data efficiency hinders their practical applications. In this work, we propose LayoutPrompter, which leverages large language models (LLMs) to address the above problems through in-context learning. LayoutPrompter is made up of three key components, namely input-output serialization, dynamic exemplar selection and layout ranking. Specifically, the input-output serialization component meticulously designs the input and output formats for each layout generation task. Dynamic exemplar selection is responsible for selecting the most helpful prompting exemplars for a given input. And a layout ranker is used to pick the highest quality layout from multiple outputs of LLMs. We conduct experiments on all existing layout generation tasks using four public datasets. Despite the simplicity of our approach, experimental results show that LayoutPrompter can compete with or even outperform state-of-the-art approaches on these tasks without any model training or fine-tuning. This demonstrates the effectiveness of this versatile and training-free approach. In addition, the ablation studies show that LayoutPrompter is significantly superior to the training-based baseline in a low-data regime, further indicating the data efficiency of LayoutPrompter. Our project is available at https://github.com/microsoft/LayoutGeneration/tree/main/LayoutPrompter.