RoboGen：朝向透過生成式模擬釋放無限數據以用於自動機器人學習的方向前進

摘要

我們提出了 RoboGen，一個能夠透過生成式模擬自動學習多樣化機器人技能的生成式機器人代理。RoboGen利用了基礎和生成模型的最新進展。我們主張使用生成方案，而非直接使用或適應這些模型來生成策略或低層級動作，透過這些模型自動生成多樣化任務、場景和訓練監督，從而在最小人類監督下擴展機器人技能學習。我們的方法為機器人代理配備了自導式提議-生成-學習循環：代理首先提出有趣的任務和技能來發展，然後通過以適當的空間配置填充相關對象和資產來生成相應的模擬環境。之後，代理將提出的高層任務分解為子任務，選擇最佳的學習方法（強化學習、運動規劃或軌跡優化），生成所需的訓練監督，然後學習策略以獲取提出的技能。我們的工作旨在提取大規模模型中嵌入的廣泛多樣化知識，並將其轉移到機器人領域。我們的完全生成式流程可以被重複查詢，生成與多樣化任務和環境相關的技能演示的無盡流。

English

We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. RoboGen leverages the latest advancements in foundation and generative models. Instead of directly using or adapting these models to produce policies or low-level actions, we advocate for a generative scheme, which uses these models to automatically generate diversified tasks, scenes, and training supervisions, thereby scaling up robotic skill learning with minimal human supervision. Our approach equips a robotic agent with a self-guided propose-generate-learn cycle: the agent first proposes interesting tasks and skills to develop, and then generates corresponding simulation environments by populating pertinent objects and assets with proper spatial configurations. Afterwards, the agent decomposes the proposed high-level task into sub-tasks, selects the optimal learning approach (reinforcement learning, motion planning, or trajectory optimization), generates required training supervision, and then learns policies to acquire the proposed skill. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics. Our fully generative pipeline can be queried repeatedly, producing an endless stream of skill demonstrations associated with diverse tasks and environments.

RoboGen：朝向透過生成式模擬釋放無限數據以用於自動機器人學習的方向前進

RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation

摘要

Support