从像素到文字——迈向大规模原生统一视觉模型

摘要

当前视觉语言模型通常通过多阶段对齐，将独立的图像编码器和语言解码器拼接在一起，这种模块化框架不可避免地导致像素级信号在帧间碎片化，并使早期像素-词汇交互分散。与此同时，原生视觉语言模型虽然在单张图像上表现优异，但在多图像、视频理解及空间智能方面仍鲜有探索。为此，我们提出NEO-ov——一种原生基础模型，无需任何外部编码器、辅助适配器或事后融合，即可端到端地学习跨帧与像素-词汇对应关系。通过彻底消除模块边界，NEO-ov使模型内部自然涌现细粒度且统一的时空建模能力。值得注意的是，NEO-ov在精细视觉感知方面表现卓越，同时大幅缩小了与模块化方案的性能差距，验证了原生"单视觉"架构不仅可行，更可在规模化下具备竞争力。除实证性能外，我们系统揭示了架构分析细节与详细训练方案，以推动后续原生多模态建模发展。相关代码与模型已开源发布：https://github.com/EvolvingLMMs-Lab/NEO。

English

Current vision-language models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment, a modular framework that inevitably fragments pixel-level signals across frames and scatters early pixel-word interactions. In parallel, native VLMs, despite impressive performance on single images, remain largely unexplored in multi-image, video understanding, and spatial intelligence. Hence, we introduce NEO-ov, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end, without any external encoders, auxiliary adapters, or post-hoc fusion. By eliminating module boundaries entirely, NEO-ov enables fine-grained and unified spatiotemporal modeling to emerge natively inside the model. Notably, NEO-ov largely narrows the gap to modular counterparts while excelling at fine-grained visual perception, validating that native "one-vision" architectures are not only feasible but competitive at scale. Beyond empirical performance, we unveil systematic architectural analyses and detailed training recipes to facilitate subsequent native multimodal modeling. Our code and models are publicly available at: https://github.com/EvolvingLMMs-Lab/NEO.