将图像作为集合进行分词

摘要

本文提出了一种基于集合的标记化与分布建模的全新图像生成范式。不同于传统方法将图像序列化为固定位置的潜在编码并采用统一压缩率，我们引入了一种无序的标记集合表示法，能够根据区域语义复杂度动态分配编码容量。这种标记集合（TokenSet）增强了全局上下文聚合能力，并提高了对局部扰动的鲁棒性。针对离散集合建模这一关键挑战，我们设计了一种双向转换机制，将集合双射地转换为具有求和约束的定长整数序列。此外，我们提出了固定和离散扩散（Fixed-Sum Discrete Diffusion）——首个同时处理离散值、固定序列长度与求和不变性的框架，实现了有效的集合分布建模。实验结果表明，我们的方法在语义感知表示与生成质量上均展现出优越性。我们的创新，涵盖新颖的表示与建模策略，推动了视觉生成超越传统的序列标记范式。我们的代码与模型已公开于https://github.com/Gengzigang/TokenSet。

English

This paper proposes a fundamentally new paradigm for image generation through set-based tokenization and distribution modeling. Unlike conventional methods that serialize images into fixed-position latent codes with a uniform compression ratio, we introduce an unordered token set representation to dynamically allocate coding capacity based on regional semantic complexity. This TokenSet enhances global context aggregation and improves robustness against local perturbations. To address the critical challenge of modeling discrete sets, we devise a dual transformation mechanism that bijectively converts sets into fixed-length integer sequences with summation constraints. Further, we propose Fixed-Sum Discrete Diffusion--the first framework to simultaneously handle discrete values, fixed sequence length, and summation invariance--enabling effective set distribution modeling. Experiments demonstrate our method's superiority in semantic-aware representation and generation quality. Our innovations, spanning novel representation and modeling strategies, advance visual generation beyond traditional sequential token paradigms. Our code and models are publicly available at https://github.com/Gengzigang/TokenSet.

将图像作为集合进行分词

Tokenize Image as a Set

摘要

Summary

Support

Support