视频生成模型是通用视觉学习器

摘要

在下一个词元预测的驱动下，自然语言处理从任务专用模型转向了强大的通用基础模型。那么，在计算机视觉中，实现通用模型所需的等效催化剂是什么？本文认为，大规模的文本到视频生成为计算机视觉提供了一个强有力的预训练范式，为通用视觉智能提供了必要的时空先验、视觉-语言对齐以及可扩展性。我们提出GenCeption，利用预训练的视频生成扩散骨干网络来定义一个前馈感知模型，该模型能够根据文本指令执行各种视觉任务。实验结果表明，GenCeption在深度估计、表面法线估计、相机位姿估计、表达式指代分割及3D关键点预测等一系列任务中达到了最先进性能，通常能匹配甚至超越专用模型（如DepthAnything3、SAM3、D4RT、VGGT-Omega、Sapiens、David、Genmo和Lotus-2）。此外，在可比设置下，视频生成预训练骨干网络优于其他预训练范式（如V-JEPA和Video MAE）。重要的是，GenCeption展现出初步的数据与模型缩放特性，以及卓越的数据效率——使用比D4RT和VGGT-Omega少7到500倍的训练数据即可达到与这些领先模型相当的性能。最后，GenCeption还表现出引人注目的涌现行为：仅使用合成人体视频训练的模型能够泛化到真实世界影像以及分布之外的目标类别（例如动物和机器人）。这些发现表明，视频生成不仅仅是一种合成工具，更是通向物理世界通用视觉智能的基础路径。项目页面：https://genception.github.io

English

Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a general-purpose model in computer vision? In this paper, we contend that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability required for general visual intelligence. We introduce GenCeption, which leverages a pre-trained video generative diffusion backbone to define a feed-forward perception model, capable of performing various vision tasks steered by text instructions. Empirical results demonstrate that GenCeption achieves state-of-the-art performance across a diverse suite of tasks, including depth, surface normal, and camera pose estimation, expression-referring segmentation, and 3D keypoint prediction, often matching or surpassing specialized models (e.g. DepthAnything3, SAM3, D4RT, VGGT-Omega, Sapiens, David, Genmo, and Lotus-2). Furthermore, the video generative pretrained backbone outperforms alternative pretraining paradigms (e.g., V-JEPA, and Video MAE) under comparable settings. Importantly, GenCeption exhibits preliminary data and model scaling properties along with exceptional data efficiency, where it achieves comparable performance with leading models like D4RT and VGGT-Omega with 7 to 500 less training data. Finally, GenCeption also exhibits intriguing emergent behaviors: a model trained exclusively on synthetic human videos generalizes to real-world footage and out-of-distribution object categories (e.g., animals and robots). These findings suggest that video generation is not merely a synthesis tool, but a foundational path toward generalist vision intelligence for the physical world. Project page: https://genception.github.io

视频生成模型是通用视觉学习器

Video Generation Models are General-Purpose Vision Learners

摘要

Support