CGB-DM：基于Transformer扩散模型的内容和图形平衡布局生成

CGB-DM: Content and Graphic Balance Layout Generation with Transformer-based Diffusion Model

July 21, 2024

作者: Yu Li, Yifan Chen, Gongye Liu, Jie Wu, Yujiu Yang

cs.AI

摘要

布局生成是智能设计的基础任务，需要整合视觉美学和内容传递的和谐表达。然而，现有方法在生成精确且视觉上吸引人的布局方面仍面临挑战，包括布局之间的阻塞、重叠或空间错位等问题，这些问题与图形布局的空间结构密切相关。我们发现这些方法过分关注内容信息，缺乏对布局空间结构的约束，导致学习内容感知和图形感知特征失衡。为了解决这个问题，我们提出了基于Transformer扩散模型的内容和图形平衡布局生成（CGB-DM）。具体而言，我们首先设计了一个调节器，平衡了预测的内容和图形权重，克服了更多关注画布上内容的倾向。其次，我们引入了显著性边界框的图形约束，进一步增强了布局表示和图像之间几何特征的对齐。此外，我们采用了Transformer为骨干的扩散模型，其强大的生成能力确保了布局生成的质量。大量实验结果表明，我们的方法在定量和定性评估中均取得了最先进的性能。我们的模型框架还可以扩展到其他图形设计领域。

English

Layout generation is the foundation task of intelligent design, which requires the integration of visual aesthetics and harmonious expression of content delivery. However, existing methods still face challenges in generating precise and visually appealing layouts, including blocking, overlap, or spatial misalignment between layouts, which are closely related to the spatial structure of graphic layouts. We find that these methods overly focus on content information and lack constraints on layout spatial structure, resulting in an imbalance of learning content-aware and graphic-aware features. To tackle this issue, we propose Content and Graphic Balance Layout Generation with Transformer-based Diffusion Model (CGB-DM). Specifically, we first design a regulator that balances the predicted content and graphic weight, overcoming the tendency of paying more attention to the content on canvas. Secondly, we introduce a graphic constraint of saliency bounding box to further enhance the alignment of geometric features between layout representations and images. In addition, we adapt a transformer-based diffusion model as the backbone, whose powerful generation capability ensures the quality in layout generation. Extensive experimental results indicate that our method has achieved state-of-the-art performance in both quantitative and qualitative evaluations. Our model framework can also be expanded to other graphic design fields.