立方體：Roblox視角下的三維智能

Cube: A Roblox View of 3D Intelligence

March 19, 2025

作者: Foundation AI Team, Kiran Bhat, Nishchaie Khanna, Karun Channa, Tinghui Zhou, Yiheng Zhu, Xiaoxia Sun, Charles Shang, Anirudh Sudarshan, Maurice Chu, Daiqing Li, Kangle Deng, Jean-Philippe Fauconnier, Tijmen Verhulsdonck, Maneesh Agrawala, Kayvon Fatahalian, Alexander Weiss, Christian Reiser, Ravi Kiran Chirravuri, Ravali Kandur, Alejandro Pelaez, Akash Garg, Michael Palleschi, Jessica Wang, Skylar Litz, Leon Liu, Anying Li, David Harmon, Derek Liu, Liangjun Feng, Denis Goupil, Lukas Kuczynski, Jihyun Yoon, Naveen Marri, Peiye Zhuang, Yinan Zhang, Brian Yin, Haomiao Jiang, Marcel van Workum, Thomas Lane, Bryce Erickson, Salil Pathare, Kyle Price, Anupam Singh, David Baszucki

cs.AI

摘要

基於海量數據訓練的基礎模型在文本、圖像、音頻和視頻領域展現了卓越的推理與生成能力。在Roblox，我們的目標是構建一個面向3D智能的基礎模型，該模型能夠支持開發者創建Roblox體驗的各個方面，從生成3D物體和場景，到為動畫角色進行綁定，再到生成描述物體行為的程序腳本。我們討論了此類3D基礎模型的三個關鍵設計需求，並展示了我們在構建此模型過程中的第一步。我們預計3D幾何形狀將成為核心數據類型，並介紹了我們針對3D形狀分詞器的解決方案。我們展示了如何將此分詞方案應用於文本到形狀生成、形狀到文本生成以及文本到場景生成的應用中。我們還演示了這些應用如何與現有的大型語言模型（LLMs）協作，進行場景分析與推理。最後，我們概述了構建一個完全統一的3D智能基礎模型的路徑。

English

Foundation models trained on vast amounts of data have demonstrated remarkable reasoning and generation capabilities in the domains of text, images, audio and video. Our goal at Roblox is to build such a foundation model for 3D intelligence, a model that can support developers in producing all aspects of a Roblox experience, from generating 3D objects and scenes to rigging characters for animation to producing programmatic scripts describing object behaviors. We discuss three key design requirements for such a 3D foundation model and then present our first step towards building such a model. We expect that 3D geometric shapes will be a core data type and describe our solution for 3D shape tokenizer. We show how our tokenization scheme can be used in applications for text-to-shape generation, shape-to-text generation and text-to-scene generation. We demonstrate how these applications can collaborate with existing large language models (LLMs) to perform scene analysis and reasoning. We conclude with a discussion outlining our path to building a fully unified foundation model for 3D intelligence.