Orca：世界在你心中

摘要

我們介紹Orca，這是一個通用世界基礎模型的初步實例。Orca從多模態世界訊號中學習統一的潛在世界空間，並透過多模態讀取介面將其呈現。與其專注於孤立的下一詞元、下一幀或下一動作預測，我們以「下一狀態預測」建模為核心，提供一條統一狀態轉換建模的路徑，以理解、預測並作用於世界。Orca透過兩種互補範式學習：無意識學習從連續影片中捕捉密集的自然狀態轉換，而有意識學習則透過語言描述的事件和VQA監督來建模稀疏且有意義的狀態轉換。在預訓練階段，我們建構大規模的世界學習資料庫，包含12.5萬小時的影片資料和1.6億條事件註解。預訓練後，Orca學習到統一的潛在世界空間。為了檢驗所學到的潛在表徵是否支援下游任務，我們透過三種代表性的下游讀取任務進行評估：文字生成、影像預測和具身動作生成。Orca的骨幹網路被凍結，僅訓練輕量級的模態特定解碼器。實驗證明了所提出範式的可擴展性，並驗證更強的潛在世界表徵能帶來更強的下游讀取能力。Orca的表現超越了類似規模的專業基線模型。這些結果顯示，Orca作為通用世界基礎模型，為理解、預測並作用於世界提供了一種有前景的方法。最後，我們討論了目前的局限性，旨在為學界提供有價值的見解與啟發。

English

We introduce Orca, an initial instantiation of a general world foundation model. Orca learns a unified world latent space from multimodal world signals and exposes it through multimodal readout interfaces. Rather than optimizing isolated next-token, next-frame, or next-action prediction, we are centered on Next-State-Prediction modeling, offering a unified state-transition modeling route toward understanding, predicting, and acting upon the world. Orca learns through two complementary paradigms: unconscious learning captures dense natural state transitions from continuous videos, and conscious learning models sparse meaningful state transitions by language-described events and VQA supervision. For pre-training, we construct a large-scale world-learning inventory data, including 125K hours of video data and 160M event annotations. After pre-training, Orca learns a unified world latent space. To examine whether the learned latent supports downstream, we evaluate it by three representative downstream readouts: text generation, image prediction, and embodied action generation. Orca's backbone is frozen, and only the lightweight modality-specific decoders are trainable. Experiments show the scalability of the proposed paradigm and verify that stronger world latent enables stronger downstream readouts. Orca outperforms similar-sized specialized baselines. These results show that Orca, as a general world foundation model, presents a promising approach to understanding, predicting, and acting upon the world. Finally, we discuss the current limitations, aiming to provide useful insights and inspiration for the community.