论像素文本表征学习的设计基础

摘要

文本丰富的视觉输入要求模型能够在像素空间中直接读取、检索和压缩语言；然而，现有的像素文本编码器仍难以克服固定分辨率预训练带来的局限，并易受视觉捷径学习、视觉锚定不足以及多语言视觉文本理解困难等问题的困扰。在这项工作中，我们研究了实现稳健的视觉文本表示学习所需的基本设计原则。通过系统的受控消融实验，我们确定了四项关键要素：第一，可变的图像分辨率和渲染字体大小可为高分辨率文档泛化提供空间层面的代理信号；第二，自然图像-文本对是实现视觉锚定的基础，并能防止模型坍缩为仅依赖文本的模式；第三，布局感知渲染有助于抑制像素级捷径的习得；第四，两阶段多语言课程学习可实现有效的跨语言对齐。我们将这些原则整合为可扩展的训练方案，由此训练出 Pixel Linguist II——一种原生分辨率视觉编码器，其在 2.8 亿条训练样本上结合了动态渲染、统一对比式锚定与多语言课程学习。Pixel Linguist II 在英语、跨语言及多语言的 Visual STS 和 ViDoRe 基准上刷新了最优结果，并在 MLLM 的下游评测中带来更优表现。值得注意的是，Pixel Linguist II 在 80% 的视觉令牌压缩下依然保持稳健，展现出其在光学上下文压缩方面的巨大潜力。我们的代码与资源可在 https://github.com/Pixel-Linguist/Pixel-Linguist-II 获取。

English

Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders struggle with fixed resolution pretraining, visual shortcut learning, weak visual grounding, and multilingual visual text understanding. In this work, we investigate the fundamental design principles required for robust visual text representation learning. Through systematic controlled ablations, we identify four critical components: variable image resolutions and rendered font sizes provide spatial proxies for high-resolution document generalization; natural image-text pairs are indispensable for grounding and prevent text-only collapse; layout-aware rendering helps prevent pixel-level shortcuts; and a two-stage multilingual curriculum enables effective cross-lingual alignment. By integrating these principles into a scalable training recipe, we train Pixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unified contrastive grounding, and a multilingual curriculum over 280M training examples. Pixel Linguist II sets new state-of-the-art results on English, cross-lingual, and multilingual Visual STS and ViDoRe, while also enabling better MLLM downstream evaluation. Notably, Pixel Linguist II remains robust under 80\% visual token compression, showing great promise for optical context compression. Our code and resources are available at https://github.com/Pixel-Linguist/Pixel-Linguist-II.