RoboVerse：邁向可擴展且可泛化的機器人學習統一平台、數據集與基準

RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning

April 26, 2025

作者: Haoran Geng, Feishi Wang, Songlin Wei, Yuyang Li, Bangjun Wang, Boshi An, Charlie Tianyue Cheng, Haozhe Lou, Peihao Li, Yen-Jen Wang, Yutong Liang, Dylan Goetting, Chaoyi Xu, Haozhe Chen, Yuxi Qian, Yiran Geng, Jiageng Mao, Weikang Wan, Mingtong Zhang, Jiangran Lyu, Siheng Zhao, Jiazhao Zhang, Jialiang Zhang, Chengyang Zhao, Haoran Lu, Yufei Ding, Ran Gong, Yuran Wang, Yuxuan Kuang, Ruihai Wu, Baoxiong Jia, Carlo Sferrazza, Hao Dong, Siyuan Huang, Yue Wang, Jitendra Malik, Pieter Abbeel

cs.AI

摘要

數據規模化和標準化評估基準已推動了自然語言處理和計算機視覺領域的顯著進步。然而，機器人技術在數據規模化和建立評估協議方面面臨著獨特的挑戰。收集現實世界的數據既耗費資源又效率低下，而在現實場景中進行基準測試仍然極為複雜。合成數據和模擬提供了有前景的替代方案，但現有努力在數據質量、多樣性和基準標準化方面往往不足。為應對這些挑戰，我們引入了RoboVerse，這是一個包含模擬平台、合成數據集和統一基準的綜合框架。我們的模擬平台支持多種模擬器和機器人實體，實現了不同環境間的無縫切換。合成數據集通過多種方法構建，具有高保真物理和逼真渲染。此外，我們提出了模仿學習和強化學習的統一基準，使得跨不同泛化層次的評估成為可能。模擬平台的核心是MetaSim，這是一個將多樣模擬環境抽象為通用接口的基礎設施。它將現有模擬環境重構為模擬器無關的配置系統，以及一個對齊不同模擬器功能的API，如啟動模擬環境、加載帶有初始狀態的資產、推進物理引擎等。這種抽象確保了互操作性和可擴展性。全面的實驗表明，RoboVerse提升了模仿學習、強化學習、世界模型學習以及模擬到現實遷移的性能。這些結果驗證了我們數據集和基準的可靠性，確立了RoboVerse作為推進機器人學習的強大解決方案。

English

Data scaling and standardized evaluation benchmarks have driven significant advances in natural language processing and computer vision. However, robotics faces unique challenges in scaling data and establishing evaluation protocols. Collecting real-world data is resource-intensive and inefficient, while benchmarking in real-world scenarios remains highly complex. Synthetic data and simulation offer promising alternatives, yet existing efforts often fall short in data quality, diversity, and benchmark standardization. To address these challenges, we introduce RoboVerse, a comprehensive framework comprising a simulation platform, a synthetic dataset, and unified benchmarks. Our simulation platform supports multiple simulators and robotic embodiments, enabling seamless transitions between different environments. The synthetic dataset, featuring high-fidelity physics and photorealistic rendering, is constructed through multiple approaches. Additionally, we propose unified benchmarks for imitation learning and reinforcement learning, enabling evaluation across different levels of generalization. At the core of the simulation platform is MetaSim, an infrastructure that abstracts diverse simulation environments into a universal interface. It restructures existing simulation environments into a simulator-agnostic configuration system, as well as an API aligning different simulator functionalities, such as launching simulation environments, loading assets with initial states, stepping the physics engine, etc. This abstraction ensures interoperability and extensibility. Comprehensive experiments demonstrate that RoboVerse enhances the performance of imitation learning, reinforcement learning, world model learning, and sim-to-real transfer. These results validate the reliability of our dataset and benchmarks, establishing RoboVerse as a robust solution for advancing robot learning.