StrokeNUWA：用于矢量图形合成的笔画标记化

摘要

为了利用LLMs进行视觉合成，传统方法通过专门的视觉模块将光栅图像信息转换为离散的网格标记，同时破坏了模型捕捉视觉场景真实语义表示的能力。本文认为，图像的另一种表示形式，矢量图形，可以通过实现更自然和语义连贯的图像信息分割，有效地克服这一局限。因此，我们介绍了StrokeNUWA，这是一项开创性工作，探索了在矢量图形上更好的视觉表示“笔画标记”，这种表示方式在视觉语义方面丰富，与LLMs自然兼容，并且高度压缩。搭载笔画标记，StrokeNUWA在矢量图形生成任务中可以显著超越传统的基于LLMs和基于优化的方法在各种指标上的表现。此外，StrokeNUWA在推断速度上实现了高达94倍的加速，具有出色的SVG代码压缩比达到6.9%。

English

To leverage LLMs for visual synthesis, traditional methods convert raster image information into discrete grid tokens through specialized visual modules, while disrupting the model's ability to capture the true semantic representation of visual scenes. This paper posits that an alternative representation of images, vector graphics, can effectively surmount this limitation by enabling a more natural and semantically coherent segmentation of the image information. Thus, we introduce StrokeNUWA, a pioneering work exploring a better visual representation ''stroke tokens'' on vector graphics, which is inherently visual semantics rich, naturally compatible with LLMs, and highly compressed. Equipped with stroke tokens, StrokeNUWA can significantly surpass traditional LLM-based and optimization-based methods across various metrics in the vector graphic generation task. Besides, StrokeNUWA achieves up to a 94x speedup in inference over the speed of prior methods with an exceptional SVG code compression ratio of 6.9%.

StrokeNUWA：用于矢量图形合成的笔画标记化

StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis

摘要

Support