RoboGen：通过生成式模拟实现自动机器人学习的无限数据释放

摘要

我们提出了RoboGen，这是一个生成式机器人代理，通过生成式模拟自动学习各种机器人技能。RoboGen利用了基础和生成模型的最新进展。我们主张采用生成式方案，而不是直接使用或调整这些模型来生成策略或低级动作，通过使用这些模型自动生成多样化的任务、场景和训练监督，从而在最小人类监督下扩展机器人技能学习。我们的方法为机器人代理配备了一个自主提出-生成-学习循环：代理首先提出有趣的任务和技能以开发，然后通过在适当的空间配置中填充相关对象和资产来生成相应的模拟环境。随后，代理将提出的高级任务分解为子任务，选择最佳学习方法（强化学习、运动规划或轨迹优化），生成所需的训练监督，然后学习获取提出的技能的策略。我们的工作旨在提取大规模模型中蕴含的广泛多样的知识，并将其转移到机器人领域。我们的完全生成式流水线可以反复查询，生成与各种任务和环境相关的技能演示的无尽流。

English

We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. RoboGen leverages the latest advancements in foundation and generative models. Instead of directly using or adapting these models to produce policies or low-level actions, we advocate for a generative scheme, which uses these models to automatically generate diversified tasks, scenes, and training supervisions, thereby scaling up robotic skill learning with minimal human supervision. Our approach equips a robotic agent with a self-guided propose-generate-learn cycle: the agent first proposes interesting tasks and skills to develop, and then generates corresponding simulation environments by populating pertinent objects and assets with proper spatial configurations. Afterwards, the agent decomposes the proposed high-level task into sub-tasks, selects the optimal learning approach (reinforcement learning, motion planning, or trajectory optimization), generates required training supervision, and then learns policies to acquire the proposed skill. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics. Our fully generative pipeline can be queried repeatedly, producing an endless stream of skill demonstrations associated with diverse tasks and environments.

RoboGen：通过生成式模拟实现自动机器人学习的无限数据释放

RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation

摘要

Support