ARM：一种具有统一离散表示的自回归大型多模态模型

摘要

本文介绍ARM——一种基于离散表示的自回归模型，它在下一个词元预测框架内统一了图像理解、生成与编辑能力。ARM的构建基于三项核心工作：首先，我们训练了一个离散语义视觉分词器，可将图像映射为紧凑的词元序列。该分词器通过多目标监督学习，同时促进语义判别性、语言对齐和忠实重建，从而在共享隐空间中支持多样化任务。在此基础上，我们在大规模文本与图像词元序列上训练了一个70亿参数的自回归模型，无缝开发了视觉-语言感知与生成能力。最后，为进一步优化文本到图像生成与指令引导编辑中符合偏好的行为，ARM应用强化学习（RL）来优化任务级目标，如视觉质量、指令遵循度和编辑一致性。令人惊讶的是，结果表明RL不仅显著提升了目标任务性能（例如，WISE综合得分从0.50提升至0.56，GEdit-Bench-EN的G_O从5.75提升至6.68），还诱导了文本到图像生成与编辑之间的跨任务协同效应。这些发现共同表明，自回归建模与强大表征及偏好优化相结合，可作为多模态智能的可扩展基础。代码：https://github.com/wdrink/ARM。

English

This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-token prediction framework. ARM is built on three efforts: first, we train a discrete semantic visual tokenizer that maps images into compact token sequences. Our tokenizer is supervised with multiple objectives that jointly promote semantic discriminability, language alignment and faithful reconstruction, thereby supporting diverse tasks in a shared latent space. With this, we train a 7B autoregressive model over large-scale text and image token sequences, seamlessly developing vision-language perception and generation capabilities. Finally, to further improve preference-aligned behavior for text-to-image generation and instruction-guided editing, ARM applies reinforcement learning (RL) to optimize task-level objectives such as visual quality, instruction adherence, and edit consistency. Surprisingly, the results show that RL not only substantially improves performance on the target tasks (e.g., raising WISE overall from 0.50 to 0.56, GEdit-Bench-EN G_O from 5.75 to 6.68), but also induces cross-task synergy between text-to-image generation and editing. Collectively, these findings highlight autoregressive modeling, when paired with strong representations and preference optimization, as a scalable foundation for multimodal intelligence. Code: https://github.com/wdrink/ARM.