并行展开近似用于像素空间自回归图像生成

摘要

像素空间连续标记自回归（AR）生成直接对图像建模为原始像素块的序列，避免了离散标记化或单独预训练的标记器。然而，它面临两个相互关联的挑战：高维块生成导致较大的单步误差，而教师强制训练造成训练与推理之间的差距，使得这些误差在自回归步骤中不断累积。现有的修复方法如x预测和输入噪声注入仅能部分缓解这些问题。精确的展开训练虽然更符合推理时的条件，但由于顺序采样的速度过慢而不切实际。我们提出了并行展开近似（PRA），这是一个可扩展的框架，能够联合解决这两个挑战。PRA生成低维中间状态而非高维像素块，然后通过像素解码器将它们映射回像素空间标记，从而保留了像素输入、像素输出的自回归接口。它还通过推理时使用的相同中间状态到像素路径，独立地在各个位置构建类似推理的像素输入，近似推理时展开过程中遇到的像素反馈接口，同时保持并行的教师强制训练。在256×256分辨率下的类条件ImageNet-1K生成任务中，具有135M参数的PRA-S实现了2.58的FID，超过了此前十亿级像素空间自回归模型3.60的结果。将规模扩展到具有511M参数的PRA-L，FID进一步提升至1.94，在像素空间自回归模型中确立了新的最优水平。除了生成任务，PRA在ImageNet分类探测准确率上也优于其他自回归和扩散基线模型，表明其在像素空间图像生成与理解统一方面的潜力。

English

Pixel-space continuous-token autoregressive (AR) generation directly models images as sequences of raw pixel patches, avoiding discrete tokenization or a separately pretrained tokenizer. However, it faces coupled challenges: high-dimensional patch generation causes large single-step errors, and teacher-forced training creates a train--inference gap that makes these errors accumulate across AR steps. Existing fixes such as x-prediction and input noise injection only partially mitigate these issues. Exact rollout training better matches inference-time conditions, but is impractical due to prohibitively slow sequential sampling. We propose Parallel Rollout Approximation (PRA), a scalable framework that addresses both challenges jointly. PRA generates low-dimensional intermediate states instead of high-dimensional pixel patches, then maps them back to pixel-space tokens with a pixel decoder, preserving a pixel-in, pixel-out AR interface. It also constructs inference-like pixel inputs through the same intermediate-state-to-pixel path used at inference, independently across positions, approximating the pixel-feedback interface encountered during inference-time rollout while retaining parallel teacher-forced training. On class-conditional ImageNet-1K generation at 256times256 resolution, PRA-S with 135M parameters achieves an FID of 2.58, surpassing the previous billion-scale pixel-space AR result of 3.60. Scaling to PRA-L with 511M parameters further improves FID to 1.94, establishing a new state of the art among pixel-space AR models. Beyond generation, PRA achieves higher ImageNet classification probing accuracy than other AR and diffusion baselines, suggesting its potential for unified pixel-space image generation and understanding.