将数据置于离线多智体强化学习的中心

摘要

离线多智体强化学习（MARL）是一个令人兴奋的研究方向，利用静态数据集为多智体系统寻找最优控制策略。虽然这一领域从本质上是数据驱动的，但迄今为止的努力忽视了数据，而专注于实现最先进的结果。我们首先通过调查文献来证实这一观点，展示大多数作品如何生成自己的数据集，缺乏一致的方法论，并提供有关这些数据集特征的稀缺信息。然后，我们展示忽视数据性质为何是有问题的，通过突出示例说明算法性能与使用的数据集密切相关，需要在该领域进行实验的共同基础。作为回应，我们迈出了一大步，以改善离线MARL中的数据使用和数据意识，具体包括三个关键贡献：（1）明确生成新数据集的指南；（2）对80多个现有数据集进行标准化，存储在一个公开可用的存储库中，使用一致的存储格式和易于使用的API；以及（3）一套分析工具，让我们更好地了解这些数据集，促进进一步发展。

English

Offline multi-agent reinforcement learning (MARL) is an exciting direction of research that uses static datasets to find optimal control policies for multi-agent systems. Though the field is by definition data-driven, efforts have thus far neglected data in their drive to achieve state-of-the-art results. We first substantiate this claim by surveying the literature, showing how the majority of works generate their own datasets without consistent methodology and provide sparse information about the characteristics of these datasets. We then show why neglecting the nature of the data is problematic, through salient examples of how tightly algorithmic performance is coupled to the dataset used, necessitating a common foundation for experiments in the field. In response, we take a big step towards improving data usage and data awareness in offline MARL, with three key contributions: (1) a clear guideline for generating novel datasets; (2) a standardisation of over 80 existing datasets, hosted in a publicly available repository, using a consistent storage format and easy-to-use API; and (3) a suite of analysis tools that allow us to understand these datasets better, aiding further development.

将数据置于离线多智体强化学习的中心

Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning

摘要

Summary

Support

Support