POGEMA：合作多智能體導航的基準平台

POGEMA: A Benchmark Platform for Cooperative Multi-Agent Navigation

July 20, 2024

作者: Alexey Skrynnik, Anton Andreychuk, Anatolii Borzilov, Alexander Chernyavskiy, Konstantin Yakovlev, Aleksandr Panov

cs.AI

摘要

最近，多智能體強化學習（MARL）在解決各種環境中具有挑戰性的合作和競爭性多智能體問題方面取得了卓越成就，這些環境主要包括少量智能體和完全可觀察性。此外，一系列重要的與機器人相關的任務，例如多機器人導航和障礙物避免，傳統上是通過經典的不可學習方法（例如，啟發式搜索）來處理，目前建議使用基於學習或混合方法來解決。然而，在這個領域中，由於缺乏支持學習和評估的統一框架，很難說不可能進行經典方法、基於學習的方法和混合方法之間的公平比較。為此，我們介紹了POGEMA，這是一套包括快速學習環境、問題實例生成器、預定義問題集、可視化工具包和允許自動評估的基準工具的綜合工具。我們介紹並明確了一套評估協議，該協議定義了一系列基於主要評估指標（例如成功率和路徑長度）計算的與領域相關的指標，從而實現公平的多重比較。我們呈現了這種比較的結果，其中涉及各種最先進的MARL、基於搜索的方法和混合方法。

English

Multi-agent reinforcement learning (MARL) has recently excelled in solving challenging cooperative and competitive multi-agent problems in various environments with, mostly, few agents and full observability. Moreover, a range of crucial robotics-related tasks, such as multi-robot navigation and obstacle avoidance, that have been conventionally approached with the classical non-learnable methods (e.g., heuristic search) is currently suggested to be solved by the learning-based or hybrid methods. Still, in this domain, it is hard, not to say impossible, to conduct a fair comparison between classical, learning-based, and hybrid approaches due to the lack of a unified framework that supports both learning and evaluation. To this end, we introduce POGEMA, a set of comprehensive tools that includes a fast environment for learning, a generator of problem instances, the collection of pre-defined ones, a visualization toolkit, and a benchmarking tool that allows automated evaluation. We introduce and specify an evaluation protocol defining a range of domain-related metrics computed on the basics of the primary evaluation indicators (such as success rate and path length), allowing a fair multi-fold comparison. The results of such a comparison, which involves a variety of state-of-the-art MARL, search-based, and hybrid methods, are presented.