RoboFactory：探索具身智能体在组合约束下的协作机制

RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints

March 20, 2025

作者: Yiran Qin, Li Kang, Xiufeng Song, Zhenfei Yin, Xiaohong Liu, Xihui Liu, Ruimao Zhang, Lei Bai

cs.AI

摘要

设计高效的具身多智能体系统对于解决跨领域的复杂现实任务至关重要。鉴于具身多智能体系统的复杂性，现有方法难以自动生成此类系统所需的安全高效训练数据。为此，我们提出了具身多智能体系统的组合约束概念，以应对具身智能体间协作带来的挑战。我们设计了针对不同类型约束的多样化接口，实现了与物理世界的无缝交互。借助组合约束及专门设计的接口，我们开发了一个面向具身多智能体系统的自动化数据收集框架，并推出了首个具身多智能体操作基准——RoboFactory。基于RoboFactory基准，我们调整并评估了模仿学习方法，分析了其在不同难度智能体任务中的表现。此外，我们还探索了多智能体模仿学习的架构与训练策略，旨在构建安全高效的具身多智能体系统。

English

Designing effective embodied multi-agent systems is critical for solving complex real-world tasks across domains. Due to the complexity of multi-agent embodied systems, existing methods fail to automatically generate safe and efficient training data for such systems. To this end, we propose the concept of compositional constraints for embodied multi-agent systems, addressing the challenges arising from collaboration among embodied agents. We design various interfaces tailored to different types of constraints, enabling seamless interaction with the physical world. Leveraging compositional constraints and specifically designed interfaces, we develop an automated data collection framework for embodied multi-agent systems and introduce the first benchmark for embodied multi-agent manipulation, RoboFactory. Based on RoboFactory benchmark, we adapt and evaluate the method of imitation learning and analyzed its performance in different difficulty agent tasks. Furthermore, we explore the architectures and training strategies for multi-agent imitation learning, aiming to build safe and efficient embodied multi-agent systems.