Orca：世界在你的心中

摘要

我们提出Orca，这是一个通用世界基础模型的初步实例化。Orca从多模态世界信号中学习统一的世界潜在空间，并通过多模态读出接口将其呈现。该方法并非专注于孤立的下一词元、下一帧或下一动作预测，而是以“下一状态预测”建模为核心，提供一条统一的状态转换建模路径，以理解、预测并作用于世界。Orca通过两种互补范式进行学习：无意识学习从连续视频中捕捉密集的自然状态转换，而有意识学习则通过语言描述的事件和VQA监督来建模稀疏的有意义状态转换。在预训练阶段，我们构建了大规模的世界学习清单数据，包括125K小时的视频数据和1.6亿条事件注释。预训练后，Orca学习到统一的世界潜在空间。为检验学习到的潜在空间是否支持下游任务，我们通过三种代表性下游读出任务进行评估：文本生成、图像预测和具身动作生成。Orca的主干网络被冻结，仅轻量级的模态特定解码器可训练。实验表明，所提出的范式具有可扩展性，并验证了更强的世界潜在空间能够带来更强的下游读出性能。Orca优于类似规模的专业基线模型。这些结果表明，作为通用世界基础模型，Orca为理解、预测和作用于世界提供了一种有前景的方法。最后，我们讨论了当前的局限性，旨在为社区提供有益的见解和启发。

English

We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multimodal readout interfaces. Rather than optimizing isolated next-token, next-frame, or next-action prediction, we are centered on Next-State-Prediction modeling, offering a unified state-transition modeling route toward understanding, predicting, and acting upon the world. Orca learns through two complementary paradigms: unconscious learning captures dense natural state transitions from continuous videos, and conscious learning models sparse meaningful state transitions by language-described events and VQA supervision. For pre-training, we construct a large-scale world-learning inventory data, including 125K hours of video data and 160M event annotations. After pre-training, Orca learns a unified world latent space. To examine whether the learned latent supports downstream, we evaluate it by three representative downstream readouts: text generation, image prediction, and embodied action generation. Orca's backbone is frozen, and only the lightweight modality-specific decoders are trainable. Experiments show the scalability of the proposed paradigm and verify that stronger world latent enables stronger downstream readouts. Orca outperforms similar-sized specialized baselines. These results show that Orca, as a general world foundation model, presents a promising approach to understanding, predicting, and acting upon the world. Finally, we discuss the current limitations, aiming to provide useful insights and inspiration for the community.