基于多智能体强化学习的电缆悬挂负载去中心化空中操控
Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning
August 2, 2025
作者: Jack Zeng, Andreu Matoses Gimenez, Eugene Vinitsky, Javier Alonso-Mora, Sihao Sun
cs.AI
摘要
本文首次提出了一种去中心化方法，利用微型飞行器（MAV）团队实现现实世界中电缆悬挂负载的六自由度操控。我们的方法采用多智能体强化学习（MARL）为每架MAV训练外环控制策略。与依赖集中式方案的最先进控制器不同，我们的策略无需全局状态、MAV间通信或邻近MAV信息。相反，智能体仅通过负载姿态观测进行隐式通信，从而实现了高度的可扩展性和灵活性。同时，该方法显著降低了推理时的计算成本，使得策略能够部署在机载设备上。此外，我们为MAV引入了一种新的动作空间设计，采用线性加速度和机体角速率。这一选择与鲁棒的低级控制器相结合，确保了在动态三维运动中由电缆张力引起的显著不确定性下，仍能实现可靠的仿真到现实的迁移。我们通过多种现实世界实验验证了该方法，包括在负载模型不确定性下的全姿态控制，展示了与最先进的集中式方法相当的目标点跟踪性能。我们还展示了具有异构控制策略的智能体之间的协作能力，以及对一架MAV完全失联的鲁棒性。实验视频请访问：https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl
English
This paper presents the first decentralized method to enable real-world 6-DoF
manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles
(MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train
an outer-loop control policy for each MAV. Unlike state-of-the-art controllers
that utilize a centralized scheme, our policy does not require global states,
inter-MAV communications, nor neighboring MAV information. Instead, agents
communicate implicitly through load pose observations alone, which enables high
scalability and flexibility. It also significantly reduces computing costs
during inference time, enabling onboard deployment of the policy. In addition,
we introduce a new action space design for the MAVs using linear acceleration
and body rates. This choice, combined with a robust low-level controller,
enables reliable sim-to-real transfer despite significant uncertainties caused
by cable tension during dynamic 3D motion. We validate our method in various
real-world experiments, including full-pose control under load model
uncertainties, showing setpoint tracking performance comparable to the
state-of-the-art centralized method. We also demonstrate cooperation amongst
agents with heterogeneous control policies, and robustness to the complete
in-flight loss of one MAV. Videos of experiments:
https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl