弹性决策转换器

摘要

本文介绍了弹性决策Transformer（EDT），它是现有决策Transformer（DT）及其变体的重大进展。尽管DT声称能够生成最佳轨迹，但经验证据表明它在轨迹拼接方面存在困难，这是一个涉及从一组次优轨迹中提取最佳或接近最佳轨迹的过程。所提出的EDT通过在测试时的动作推断过程中促进轨迹拼接，通过调整DT中保留的历史长度来实现。此外，EDT通过在先前轨迹最佳时保留较长的历史，而在次优时保留较短的历史来优化轨迹，使其能够与更优轨迹“拼接”。广泛的实验表明，EDT能够弥合基于DT和Q学习的方法之间的性能差距。特别是，在D4RL运动基准和Atari游戏的多任务制度中，EDT表现优于基于Q学习的方法。视频可在以下链接找到：https://kristery.github.io/edt/

English

This paper introduces Elastic Decision Transformer (EDT), a significant advancement over the existing Decision Transformer (DT) and its variants. Although DT purports to generate an optimal trajectory, empirical evidence suggests it struggles with trajectory stitching, a process involving the generation of an optimal or near-optimal trajectory from the best parts of a set of sub-optimal trajectories. The proposed EDT differentiates itself by facilitating trajectory stitching during action inference at test time, achieved by adjusting the history length maintained in DT. Further, the EDT optimizes the trajectory by retaining a longer history when the previous trajectory is optimal and a shorter one when it is sub-optimal, enabling it to "stitch" with a more optimal trajectory. Extensive experimentation demonstrates EDT's ability to bridge the performance gap between DT-based and Q Learning-based approaches. In particular, the EDT outperforms Q Learning-based methods in a multi-task regime on the D4RL locomotion benchmark and Atari games. Videos are available at: https://kristery.github.io/edt/