ALPINE：揭示自回归学习在语言模型中的规划能力

ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models

May 15, 2024

作者: Siwei Wang, Yifei Shen, Shi Feng, Haoran Sun, Shang-Hua Teng, Wei Chen

cs.AI

摘要

本文介绍了我们的ALPINE项目的研究结果，ALPINE代表“Autoregressive Learning for Planning In NEtworks”。ALPINE项目通过自回归学习机制对基于Transformer的语言模型中规划能力的发展进行了理论研究，旨在识别它们规划能力中的潜在限制。我们将规划抽象为一项网络路径查找任务，其目标是从指定的源节点生成到指定目标节点的有效路径。在表达能力方面，我们展示了Transformer能够通过将邻接矩阵和可达性矩阵嵌入其权重中来执行路径查找。我们对Transformer基于梯度的学习动态进行的理论分析揭示了Transformer能够学习邻接矩阵和有限形式的可达性矩阵。这些理论观点随后通过实验证实，实验证明Transformer确实学习了邻接矩阵和不完整的可达性矩阵，这与我们理论分析中的预测一致。此外，当将我们的方法应用于名为Blocksworld的现实世界规划基准时，我们的观察结果保持一致。我们的理论和实证分析进一步揭示了Transformer在路径查找中的潜在限制：它无法通过传递性识别可达性关系，因此在需要路径串联生成路径时会失败。总之，我们的研究结果揭示了自回归学习的内部机制如何实现网络规划。这项研究可能有助于我们了解其他相关领域的一般规划能力。

English

In this paper, we present the findings of our Project ALPINE which stands for ``Autoregressive Learning for Planning In NEtworks." Project ALPINE initiates a theoretical investigation into the development of planning capabilities in Transformer-based language models through their autoregressive learning mechanisms, aiming to identify any potential limitations in their planning abilities. We abstract planning as a network path-finding task where the objective is to generate a valid path from a specified source node to a designated target node. In terms of expressiveness, we show that the Transformer is capable of executing path-finding by embedding the adjacency and reachability matrices within its weights. Our theoretical analysis of the gradient-based learning dynamic of the Transformer reveals that the Transformer is capable of learning both the adjacency matrix and a limited form of the reachability matrix. These theoretical insights are then validated through experiments, which demonstrate that the Transformer indeed learns the adjacency matrix and an incomplete reachability matrix, which aligns with the predictions made in our theoretical analysis. Additionally, when applying our methodology to a real-world planning benchmark, called Blocksworld, our observations remain consistent. Our theoretical and empirical analyses further unveil a potential limitation of Transformer in path-finding: it cannot identify reachability relationships through transitivity, and thus would fail when path concatenation is needed to generate a path. In summary, our findings shed new light on how the internal mechanisms of autoregressive learning enable planning in networks. This study may contribute to our understanding of the general planning capabilities in other related domains.