Make-An-Agent: 一個具有行為提示擴散的通用策略網路生成器

Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusion

July 15, 2024

作者: Yongyuan Liang, Tingqiang Xu, Kaizhe Hu, Guangqi Jiang, Furong Huang, Huazhe Xu

cs.AI

摘要

我們是否可以僅使用一個所需行為示範來生成代理的控制策略，就像從文字描述創建圖像一樣輕鬆？在本文中，我們提出了Make-An-Agent，一種新穎的政策參數生成器，利用條件擴散模型的強大功能進行行為到政策生成。在行為嵌入的指導下，該政策生成器合成潛在參數表示，然後可以將其解碼為政策網絡。通過訓練政策網絡檢查點及其對應的軌跡，我們的生成模型展示了在多個任務上的卓越靈活性和可擴展性，並具有對未見任務的強大泛化能力，僅需少量示範作為輸入即可輸出表現良好的政策。我們展示了它在各種領域和任務上的效力和效率，包括不同目標、行為，甚至跨不同機器人操作者。除了模擬之外，我們還將Make-An-Agent生成的政策直接部署到現實世界的機器人上進行運動任務。

English

Can we generate a control policy for an agent using just one demonstration of desired behaviors as a prompt, as effortlessly as creating an image from a textual description? In this paper, we present Make-An-Agent, a novel policy parameter generator that leverages the power of conditional diffusion models for behavior-to-policy generation. Guided by behavior embeddings that encode trajectory information, our policy generator synthesizes latent parameter representations, which can then be decoded into policy networks. Trained on policy network checkpoints and their corresponding trajectories, our generation model demonstrates remarkable versatility and scalability on multiple tasks and has a strong generalization ability on unseen tasks to output well-performed policies with only few-shot demonstrations as inputs. We showcase its efficacy and efficiency on various domains and tasks, including varying objectives, behaviors, and even across different robot manipulators. Beyond simulation, we directly deploy policies generated by Make-An-Agent onto real-world robots on locomotion tasks.