像素空間自迴歸圖像生成的並行展開近似

摘要

像素空間連續標記自迴歸生成直接將影像建模為原始像素區塊的序列，避免了離散標記化或需另外預訓練的標記器。然而，此方法面臨兩項耦合挑戰：高維度區塊生成導致單步誤差過大，且教師強制訓練造成訓練與推理之間的落差，使這些誤差在自迴歸步驟中逐步累積。現有解決方案如x預測與輸入雜訊注入僅能部分緩解這些問題。精確展開訓練雖更貼近推理時的情境，但由於序列取樣速度過慢而不切實際。我們提出並行展開近似法，這是一個能同時應對上述兩項挑戰的可擴展框架。並行展開近似法會生成低維度中間狀態（而非高維度像素區塊），再透過像素解碼器將其映射回像素空間標記，保留了像素輸入與像素輸出的自迴歸介面。此外，該方法透過與推理時相同的中間狀態到像素路徑，在各位置上獨立建構近似推理的像素輸入，模擬推理階段展開時會遇到的像素回饋介面，同時保留平行教師強制訓練的優勢。在256×256解析度的類別條件式ImageNet-1K生成任務中，參數量為1.35億的PRA-S模型達到了2.58的FID分數，超越了先前十億參數量級像素空間自迴歸模型的3.60分數。進一步擴大至參數量5.11億的PRA-L模型，FID更提升至1.94，創下像素空間自迴歸模型的最新最佳表現。除了生成能力，PRA在ImageNet分類探測準確率上也優於其他自迴歸與擴散模型基準，顯示其在像素空間影像生成與理解整合上的潛力。

English

Pixel-space continuous-token autoregressive (AR) generation directly models images as sequences of raw pixel patches, avoiding discrete tokenization or a separately pretrained tokenizer. However, it faces coupled challenges: high-dimensional patch generation causes large single-step errors, and teacher-forced training creates a train--inference gap that makes these errors accumulate across AR steps. Existing fixes such as x-prediction and input noise injection only partially mitigate these issues. Exact rollout training better matches inference-time conditions, but is impractical due to prohibitively slow sequential sampling. We propose Parallel Rollout Approximation (PRA), a scalable framework that addresses both challenges jointly. PRA generates low-dimensional intermediate states instead of high-dimensional pixel patches, then maps them back to pixel-space tokens with a pixel decoder, preserving a pixel-in, pixel-out AR interface. It also constructs inference-like pixel inputs through the same intermediate-state-to-pixel path used at inference, independently across positions, approximating the pixel-feedback interface encountered during inference-time rollout while retaining parallel teacher-forced training. On class-conditional ImageNet-1K generation at 256times256 resolution, PRA-S with 135M parameters achieves an FID of 2.58, surpassing the previous billion-scale pixel-space AR result of 3.60. Scaling to PRA-L with 511M parameters further improves FID to 1.94, establishing a new state of the art among pixel-space AR models. Beyond generation, PRA achieves higher ImageNet classification probing accuracy than other AR and diffusion baselines, suggesting its potential for unified pixel-space image generation and understanding.