农民：基于像素的流式自回归变换器

摘要

直接对原始数据分布进行显式似然建模是机器学习领域的核心课题，通过自回归建模实现的大语言模型已展现出规模化成功。然而，在视觉像素数据上实施连续自回归建模面临着序列极长和高维空间的挑战。本文提出FARMER——一种融合归一化流与自回归模型的新型端到端生成框架，可直接基于原始像素实现可处理的似然估计与高质量图像合成。FARMER采用可逆自回归流将图像转换为潜在序列，其分布由自回归模型进行隐式建模。针对像素级建模中的冗余性与复杂性，我们提出自监督降维方案，将归一化流潜在通道划分为信息组与冗余组，从而实现更高效的自回归建模。此外，我们设计了一步蒸馏方案以显著加速推理速度，并引入基于重采样的无分类器引导算法来提升图像生成质量。大量实验表明，FARMER在提供精确似然估计和可扩展训练的同时，与现有基于像素的生成模型相比具有竞争优势。

English

Directly modeling the explicit likelihood of the raw data distribution is key topic in the machine learning area, which achieves the scaling successes in Large Language Models by autoregressive modeling. However, continuous AR modeling over visual pixel data suffer from extremely long sequences and high-dimensional spaces. In this paper, we present FARMER, a novel end-to-end generative framework that unifies Normalizing Flows (NF) and Autoregressive (AR) models for tractable likelihood estimation and high-quality image synthesis directly from raw pixels. FARMER employs an invertible autoregressive flow to transform images into latent sequences, whose distribution is modeled implicitly by an autoregressive model. To address the redundancy and complexity in pixel-level modeling, we propose a self-supervised dimension reduction scheme that partitions NF latent channels into informative and redundant groups, enabling more effective and efficient AR modeling. Furthermore, we design a one-step distillation scheme to significantly accelerate inference speed and introduce a resampling-based classifier-free guidance algorithm to boost image generation quality. Extensive experiments demonstrate that FARMER achieves competitive performance compared to existing pixel-based generative models while providing exact likelihoods and scalable training.

农民：基于像素的流式自回归变换器

FARMER: Flow AutoRegressive Transformer over Pixels

摘要

Support