Gemini Robotics: AIを物理世界に導入する

Gemini Robotics: Bringing AI into the Physical World

March 25, 2025

著者: Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, Steven Bohez, Konstantinos Bousmalis, Anthony Brohan, Thomas Buschmann, Arunkumar Byravan, Serkan Cabi, Ken Caluwaerts, Federico Casarini, Oscar Chang, Jose Enrique Chen, Xi Chen, Hao-Tien Lewis Chiang, Krzysztof Choromanski, David D'Ambrosio, Sudeep Dasari, Todor Davchev, Coline Devin, Norman Di Palo, Tianli Ding, Adil Dostmohamed, Danny Driess, Yilun Du, Debidatta Dwibedi, Michael Elabd, Claudio Fantacci, Cody Fong, Erik Frey, Chuyuan Fu, Marissa Giustina, Keerthana Gopalakrishnan, Laura Graesser, Leonard Hasenclever, Nicolas Heess, Brandon Hernaez, Alexander Herzog, R. Alex Hofer, Jan Humplik, Atil Iscen, Mithun George Jacob, Deepali Jain, Ryan Julian, Dmitry Kalashnikov, M. Emre Karagozler, Stefani Karp, Chase Kew, Jerad Kirkland, Sean Kirmani, Yuheng Kuang, Thomas Lampe, Antoine Laurens, Isabel Leal, Alex X. Lee, Tsang-Wei Edward Lee, Jacky Liang, Yixin Lin, Sharath Maddineni, Anirudha Majumdar, Assaf Hurwitz Michaely, Robert Moreno, Michael Neunert, Francesco Nori, Carolina Parada, Emilio Parisotto, Peter Pastor, Acorn Pooley, Kanishka Rao, Krista Reymann, Dorsa Sadigh, Stefano Saliceti, Pannag Sanketi, Pierre Sermanet, Dhruv Shah, Mohit Sharma, Kathryn Shea, Charles Shu, Vikas Sindhwani, Sumeet Singh, Radu Soricut, Jost Tobias Springenberg, Rachel Sterneck, Razvan Surdulescu, Jie Tan, Jonathan Tompson, Vincent Vanhoucke, Jake Varley, Grace Vesom, Giulia Vezzani, Oriol Vinyals, Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Fei Xia, Ted Xiao, Annie Xie, Jinyu Xie, Peng Xu, Sichun Xu, Ying Xu, Zhuo Xu, Yuxiang Yang, Rui Yao, Sergey Yaroshenko, Wenhao Yu, Wentao Yuan, Jingwei Zhang, Tingnan Zhang, Allan Zhou, Yuxiang Zhou

cs.AI

要旨

大規模マルチモーダルモデルの最近の進展により、デジタル領域における顕著な汎用能力が出現していますが、ロボットなどの物理的エージェントへの応用は依然として重要な課題です。本報告書では、Gemini 2.0を基盤としてロボティクス向けに特別に設計された新たなAIモデルファミリーを紹介します。私たちは、Gemini Roboticsを発表します。これは、ロボットを直接制御可能な高度なVision-Language-Action（VLA）汎用モデルです。Gemini Roboticsは、滑らかで反応的な動作を実行し、幅広い複雑な操作タスクに取り組むことができ、物体の種類や位置の変化に対して頑健であり、未見の環境に対応し、多様なオープン語彙の指示に従うことができます。追加のファインチューニングにより、Gemini Roboticsは、長期的で高度に器用なタスクの解決、わずか100回のデモンストレーションから新しい短期的タスクを学習すること、そして完全に新しいロボットの形態に適応するといった新たな能力に特化できることを示します。これは、Gemini RoboticsがGemini Robotics-ERモデルを基盤としているため可能です。Gemini Robotics-ER（Embodied Reasoning）は、Geminiのマルチモーダル推論能力を物理世界に拡張し、空間的および時間的理解を強化します。これにより、物体検出、ポインティング、軌道および把持予測、マルチビュー対応、3Dバウンディングボックス予測など、ロボティクスに関連する能力が可能になります。この新たな組み合わせが、さまざまなロボティクスアプリケーションをサポートする方法を示します。また、この新しいクラスのロボティクス基盤モデルに関連する重要な安全上の考慮事項についても議論し、対処します。Gemini Roboticsファミリーは、AIの潜在能力を物理世界で実現する汎用ロボットの開発に向けた重要な一歩を記すものです。

English

Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as robots remains a significant challenge. This report introduces a new family of AI models purposefully designed for robotics and built upon the foundation of Gemini 2.0. We present Gemini Robotics, an advanced Vision-Language-Action (VLA) generalist model capable of directly controlling robots. Gemini Robotics executes smooth and reactive movements to tackle a wide range of complex manipulation tasks while also being robust to variations in object types and positions, handling unseen environments as well as following diverse, open vocabulary instructions. We show that with additional fine-tuning, Gemini Robotics can be specialized to new capabilities including solving long-horizon, highly dexterous tasks, learning new short-horizon tasks from as few as 100 demonstrations and adapting to completely novel robot embodiments. This is made possible because Gemini Robotics builds on top of the Gemini Robotics-ER model, the second model we introduce in this work. Gemini Robotics-ER (Embodied Reasoning) extends Gemini's multimodal reasoning capabilities into the physical world, with enhanced spatial and temporal understanding. This enables capabilities relevant to robotics including object detection, pointing, trajectory and grasp prediction, as well as multi-view correspondence and 3D bounding box predictions. We show how this novel combination can support a variety of robotics applications. We also discuss and address important safety considerations related to this new class of robotics foundation models. The Gemini Robotics family marks a substantial step towards developing general-purpose robots that realizes AI's potential in the physical world.