CGB-DM：基於Transformer擴散模型的內容和圖形平衡佈局生成

CGB-DM: Content and Graphic Balance Layout Generation with Transformer-based Diffusion Model

July 21, 2024

作者: Yu Li, Yifan Chen, Gongye Liu, Jie Wu, Yujiu Yang

cs.AI

摘要

版面生成是智能設計的基礎任務，需要整合視覺美學和內容傳遞的和諧表達。然而，現有方法在生成精確且視覺上吸引人的版面方面仍面臨挑戰，包括版面之間的阻擋、重疊或空間錯位，這些問題與圖形版面的空間結構密切相關。我們發現這些方法過於注重內容信息，缺乏對版面空間結構的限制，導致在學習內容感知和圖形感知特徵之間存在不平衡。為了應對這個問題，我們提出了基於Transformer擴散模型的內容和圖形平衡版面生成（CGB-DM）。具體來說，我們首先設計一個調節器，平衡預測的內容和圖形權重，克服了更多關注畫布上內容的趨勢。其次，我們引入了一個圖形約束的显著性邊界框，進一步增強版面表示和圖像之間幾何特徵的對齊。此外，我們採用了Transformer擴散模型作為骨幹，其強大的生成能力確保了版面生成的質量。大量實驗結果表明，我們的方法在定量和定性評估中均取得了最先進的性能。我們的模型框架還可以擴展到其他圖形設計領域。

English

Layout generation is the foundation task of intelligent design, which requires the integration of visual aesthetics and harmonious expression of content delivery. However, existing methods still face challenges in generating precise and visually appealing layouts, including blocking, overlap, or spatial misalignment between layouts, which are closely related to the spatial structure of graphic layouts. We find that these methods overly focus on content information and lack constraints on layout spatial structure, resulting in an imbalance of learning content-aware and graphic-aware features. To tackle this issue, we propose Content and Graphic Balance Layout Generation with Transformer-based Diffusion Model (CGB-DM). Specifically, we first design a regulator that balances the predicted content and graphic weight, overcoming the tendency of paying more attention to the content on canvas. Secondly, we introduce a graphic constraint of saliency bounding box to further enhance the alignment of geometric features between layout representations and images. In addition, we adapt a transformer-based diffusion model as the backbone, whose powerful generation capability ensures the quality in layout generation. Extensive experimental results indicate that our method has achieved state-of-the-art performance in both quantitative and qualitative evaluations. Our model framework can also be expanded to other graphic design fields.