RoboFactory：探索具身智能体在组合约束下的协作

RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints

March 20, 2025

作者: Yiran Qin, Li Kang, Xiufeng Song, Zhenfei Yin, Xiaohong Liu, Xihui Liu, Ruimao Zhang, Lei Bai

cs.AI

摘要

設計有效的具身多智能體系統對於解決跨領域的複雜現實任務至關重要。由於具身多智能體系統的複雜性，現有方法無法自動生成此類系統的安全且高效的訓練數據。為此，我們提出了具身多智能體系統的組合約束概念，以應對具身智能體之間協作所帶來的挑戰。我們設計了針對不同類型約束的各種接口，實現與物理世界的無縫交互。利用組合約束和專門設計的接口，我們開發了一個自動化的具身多智能體系統數據收集框架，並引入了首個具身多智能體操作基準——RoboFactory。基於RoboFactory基準，我們調整並評估了模仿學習方法，並分析了其在不同難度智能體任務中的表現。此外，我們探索了多智能體模仿學習的架構和訓練策略，旨在構建安全且高效的具身多智能體系統。

English

Designing effective embodied multi-agent systems is critical for solving complex real-world tasks across domains. Due to the complexity of multi-agent embodied systems, existing methods fail to automatically generate safe and efficient training data for such systems. To this end, we propose the concept of compositional constraints for embodied multi-agent systems, addressing the challenges arising from collaboration among embodied agents. We design various interfaces tailored to different types of constraints, enabling seamless interaction with the physical world. Leveraging compositional constraints and specifically designed interfaces, we develop an automated data collection framework for embodied multi-agent systems and introduce the first benchmark for embodied multi-agent manipulation, RoboFactory. Based on RoboFactory benchmark, we adapt and evaluate the method of imitation learning and analyzed its performance in different difficulty agent tasks. Furthermore, we explore the architectures and training strategies for multi-agent imitation learning, aiming to build safe and efficient embodied multi-agent systems.