小波是自回归图像生成的全部所需

Wavelets Are All You Need for Autoregressive Image Generation

June 28, 2024

作者: Wael Mattar, Idan Levy, Nir Sharon, Shai Dekel

cs.AI

摘要

本文采用了一种新的自回归图像生成方法，其基于两个主要要素。第一个要素是小波图像编码，它允许将图像的视觉细节从粗糙到精细的顺序进行标记，方法是从最显著的小波系数的最显著位开始对信息进行排序。第二个要素是一种语言变换器的变体，其架构经过重新设计和针对在这种“小波语言”中的标记序列进行了优化。变换器学习了标记序列中的显著统计相关性，这些相关性是各种分辨率小波子带之间已知相关性的表现。我们展示了在生成过程中进行条件处理的实验结果。

English

In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visual details of an image from coarse to fine details by ordering the information starting with the most significant bits of the most significant wavelet coefficients. The second is a variant of a language transformer whose architecture is re-designed and optimized for token sequences in this 'wavelet language'. The transformer learns the significant statistical correlations within a token sequence, which are the manifestations of well-known correlations between the wavelet subbands at various resolutions. We show experimental results with conditioning on the generation process.