立方体：Roblox视角下的三维智能

Cube: A Roblox View of 3D Intelligence

March 19, 2025

作者: Foundation AI Team, Kiran Bhat, Nishchaie Khanna, Karun Channa, Tinghui Zhou, Yiheng Zhu, Xiaoxia Sun, Charles Shang, Anirudh Sudarshan, Maurice Chu, Daiqing Li, Kangle Deng, Jean-Philippe Fauconnier, Tijmen Verhulsdonck, Maneesh Agrawala, Kayvon Fatahalian, Alexander Weiss, Christian Reiser, Ravi Kiran Chirravuri, Ravali Kandur, Alejandro Pelaez, Akash Garg, Michael Palleschi, Jessica Wang, Skylar Litz, Leon Liu, Anying Li, David Harmon, Derek Liu, Liangjun Feng, Denis Goupil, Lukas Kuczynski, Jihyun Yoon, Naveen Marri, Peiye Zhuang, Yinan Zhang, Brian Yin, Haomiao Jiang, Marcel van Workum, Thomas Lane, Bryce Erickson, Salil Pathare, Kyle Price, Anupam Singh, David Baszucki

cs.AI

摘要

基于海量数据训练的基础模型在文本、图像、音频和视频领域展现出了卓越的推理与生成能力。Roblox的目标是构建一个面向3D智能的基础模型，该模型能够支持开发者打造Roblox体验的各个方面，从生成3D物体和场景，到为动画角色绑定骨骼，再到编写描述对象行为的程序脚本。我们探讨了构建此类3D基础模型的三大关键设计需求，并介绍了我们迈向这一目标的第一步。我们预期3D几何形状将成为核心数据类型，并详细阐述了我们的3D形状分词器解决方案。我们展示了该分词方案如何应用于文本到形状生成、形状到文本生成以及文本到场景生成等任务中。我们进一步演示了这些应用如何与现有的大型语言模型（LLMs）协作，进行场景分析与推理。最后，我们概述了构建一个完全统一的3D智能基础模型的未来路径。

English

Foundation models trained on vast amounts of data have demonstrated remarkable reasoning and generation capabilities in the domains of text, images, audio and video. Our goal at Roblox is to build such a foundation model for 3D intelligence, a model that can support developers in producing all aspects of a Roblox experience, from generating 3D objects and scenes to rigging characters for animation to producing programmatic scripts describing object behaviors. We discuss three key design requirements for such a 3D foundation model and then present our first step towards building such a model. We expect that 3D geometric shapes will be a core data type and describe our solution for 3D shape tokenizer. We show how our tokenization scheme can be used in applications for text-to-shape generation, shape-to-text generation and text-to-scene generation. We demonstrate how these applications can collaborate with existing large language models (LLMs) to perform scene analysis and reasoning. We conclude with a discussion outlining our path to building a fully unified foundation model for 3D intelligence.