RoboVerse：迈向可扩展与通用机器人学习的统一平台、数据集与基准

摘要

数据规模化和标准化评估基准推动了自然语言处理和计算机视觉领域的显著进步。然而，在机器人学领域，数据规模化与评估协议的建立面临独特挑战。现实世界数据的收集既耗费资源又效率低下，而在真实场景中进行基准测试则极为复杂。合成数据与仿真提供了有前景的替代方案，但现有努力在数据质量、多样性和基准标准化方面往往不足。为应对这些挑战，我们推出了RoboVerse，一个包含仿真平台、合成数据集及统一基准的综合框架。我们的仿真平台支持多种模拟器和机器人实体，实现不同环境间的无缝切换。该合成数据集通过多种方法构建，具备高保真物理特性和照片级真实感渲染。此外，我们提出了适用于模仿学习与强化学习的统一基准，支持跨不同泛化层次的评估。仿真平台的核心是MetaSim，这一基础设施将多样化的仿真环境抽象为统一接口。它重构现有仿真环境，形成模拟器无关的配置系统，以及一个对齐不同模拟器功能的API，如启动仿真环境、加载带有初始状态的资产、推进物理引擎等。这种抽象确保了互操作性和可扩展性。全面的实验表明，RoboVerse提升了模仿学习、强化学习、世界模型学习及仿真到现实迁移的性能。这些结果验证了我们数据集和基准的可靠性，确立了RoboVerse作为推动机器人学习发展的坚实解决方案。

English

Data scaling and standardized evaluation benchmarks have driven significant advances in natural language processing and computer vision. However, robotics faces unique challenges in scaling data and establishing evaluation protocols. Collecting real-world data is resource-intensive and inefficient, while benchmarking in real-world scenarios remains highly complex. Synthetic data and simulation offer promising alternatives, yet existing efforts often fall short in data quality, diversity, and benchmark standardization. To address these challenges, we introduce RoboVerse, a comprehensive framework comprising a simulation platform, a synthetic dataset, and unified benchmarks. Our simulation platform supports multiple simulators and robotic embodiments, enabling seamless transitions between different environments. The synthetic dataset, featuring high-fidelity physics and photorealistic rendering, is constructed through multiple approaches. Additionally, we propose unified benchmarks for imitation learning and reinforcement learning, enabling evaluation across different levels of generalization. At the core of the simulation platform is MetaSim, an infrastructure that abstracts diverse simulation environments into a universal interface. It restructures existing simulation environments into a simulator-agnostic configuration system, as well as an API aligning different simulator functionalities, such as launching simulation environments, loading assets with initial states, stepping the physics engine, etc. This abstraction ensures interoperability and extensibility. Comprehensive experiments demonstrate that RoboVerse enhances the performance of imitation learning, reinforcement learning, world model learning, and sim-to-real transfer. These results validate the reliability of our dataset and benchmarks, establishing RoboVerse as a robust solution for advancing robot learning.

RoboVerse：迈向可扩展与通用机器人学习的统一平台、数据集与基准

RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning

摘要

Support