小波是自回歸圖像生成的全部所需

Wavelets Are All You Need for Autoregressive Image Generation

June 28, 2024

作者: Wael Mattar, Idan Levy, Nir Sharon, Shai Dekel

cs.AI

摘要

本文採用了一種新的方法來進行自回歸圖像生成，該方法基於兩個主要要素。第一個要素是小波圖像編碼，它允許將圖像的視覺細節從粗糙到細節進行標記化，通過按照從最顯著小波係數的最顯著位開始排序信息。第二個要素是語言變換器的一個變體，其架構經過重新設計並針對這種“小波語言”的標記序列進行了優化。該變換器學習了標記序列中的顯著統計相關性，這些相關性是各種分辨率下小波子帶之間已知相關性的表現。我們展示了在生成過程中對條件進行實驗結果。

English

In this paper, we take a new approach to autoregressive image generation that is based on two main ingredients. The first is wavelet image coding, which allows to tokenize the visual details of an image from coarse to fine details by ordering the information starting with the most significant bits of the most significant wavelet coefficients. The second is a variant of a language transformer whose architecture is re-designed and optimized for token sequences in this 'wavelet language'. The transformer learns the significant statistical correlations within a token sequence, which are the manifestations of well-known correlations between the wavelet subbands at various resolutions. We show experimental results with conditioning on the generation process.