論像素文字表徵學習之設計基礎

摘要

文字豐富的視覺輸入需要能夠直接在像素空間中閱讀、檢索和壓縮語言的模型，然而現有的像素文本編碼器在固定解析度預訓練、視覺捷徑學習、弱視覺接地以及多語言視覺文本理解方面仍面臨困難。在本研究中，我們探討穩健視覺文本表徵學習所需的基本設計原則。透過系統性的受控消融實驗，我們識別出四個關鍵組成部分：可變影像解析度與渲染字體大小可提供高解析度文件泛化的空間代理；自然影像─文本配對對於接地至關重要，且能防止僅依賴文本的崩潰；感知版面配置的渲染有助於防止像素層級的捷徑；而兩階段多語言課程則能實現有效的跨語言對齊。透過將這些原則整合到可擴展的訓練配方中，我們訓練了 Pixel Linguist II——一個以原生解析度運作的視覺編碼器，採用即時渲染、統一對比接地，以及涵蓋 2.8 億訓練樣本的多語言課程。Pixel Linguist II 在英文、跨語言及多語言的 Visual STS 和 ViDoRe 基準上均創下新最先進成果，同時也促成更佳的多模態大型語言模型（MLLM）下游評估。值得注意的是，Pixel Linguist II 在 80% 視覺 token 壓縮下仍保持穩健表現，展現出光學上下文壓縮的巨大潛力。我們的程式碼與資源已公開於 https://github.com/Pixel-Linguist/Pixel-Linguist-II。

English

Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders struggle with fixed resolution pretraining, visual shortcut learning, weak visual grounding, and multilingual visual text understanding. In this work, we investigate the fundamental design principles required for robust visual text representation learning. Through systematic controlled ablations, we identify four critical components: variable image resolutions and rendered font sizes provide spatial proxies for high-resolution document generalization; natural image-text pairs are indispensable for grounding and prevent text-only collapse; layout-aware rendering helps prevent pixel-level shortcuts; and a two-stage multilingual curriculum enables effective cross-lingual alignment. By integrating these principles into a scalable training recipe, we train Pixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unified contrastive grounding, and a multilingual curriculum over 280M training examples. Pixel Linguist II sets new state-of-the-art results on English, cross-lingual, and multilingual Visual STS and ViDoRe, while also enabling better MLLM downstream evaluation. Notably, Pixel Linguist II remains robust under 80\% visual token compression, showing great promise for optical context compression. Our code and resources are available at https://github.com/Pixel-Linguist/Pixel-Linguist-II.